Novel Approaches to research and discover urban history

Abstract
The research group on four-dimensional research and communication of urban history (HistStadt4D) aims to investigate and develop methods and technologies to transfer extensive repositories of historical media and their contextual information into a threedimensional spatial model, with an additional temporal component. This will make content accessible to different target groups, researchers and the public, via a 4D browser. A location-dependent augmented-reality representation can be used as an information base, research tool, and to communicate historical knowledge. The data resources available for this research include extensive holdings of historical photographs of Dresden, which have documented the city over the decades, and digitized map collections on the Deutsche Fotothek (German photographic collection) platform. These will lay the foundation for a prototype model which will give users a virtual experience of historic parts of Dresden.

Introduction
Imagine you’re exploring the historic center of a city with its impressive town houses, churches and monuments. What if you could just use your mobile to find out about the historic buildings around you, with detailed visual information about how they were built and the story behind them, mak-ing history come alive before your eyes?

Photographs and plans are an essential source for historical research (Burke, 2003, Paul, 2006, Wohlfeil, 1986, Perez-Gomez and Pelletier, 1997) and key objects in eHumanities (Kwastek, 2014). Numerous digital image archives, containing vast numbers of photographs, have been set up in the context of digitization projects . These extensive repositories of image media are still difficult to search. It is not easy to identify sources relevant for re-search, analyze and contextualize them, or compare them with the historical original. The eHumanities research group HistStadt4D, funded by the German Fed-eral Ministry of Education and Research (BMBF) until July 2020, is investigating and developing methods and technologies for this. The junior research group consists of 14 people - including 4 post-doctoral and 4 PhD researchers. Since a focal interest is to comprehensively investigate how to enhance accessibility of large scale image repositories, researchers and research approaches originate from the humanities, geo- and information technologies as well as from educational and information studies. In contrast to adjacent projects dealing primarily with large scale linked text data as the Venice Time Machine project (2017), sources addressed by the junior group are primarily historical photographs and plans. Historical media and their con-textual information are being transferred into a 4D - 3D spatial and temporal scaled - model to support research and education on urban history. Content will be made accessible in two ways; via a 4D browser and a location-dependent augmented-reality representation. The prototype database consists of about 200,000 digitized historical photographs and plans of Dresden from the Deutsche Fotothek (German photographic collection).

Key Aspects

Usage scenarios and research values

Digital image repositories meet a wide range of needs, from research in humanities and information technologies, through museum contexts and library studies to tourist applications (Munster, 2011). Architectural historians have developed various methods of analyzing both preserved and never-built or destroyed structures in chronology and context (Brassat and Kohle, 2003). Style analysis, iconographic approaches, and art sociological methods all address structural historical questions. The technological possibilities of digital image repositories allow architecture historians to draw on a much larger stock of material, and to process and evaluate this from new perspectives. In addition, innovative software tools can be used to locate sources temporally and spatially, or to support dating, stylistic criticism, authorizations or archaeological investigations (Verstegen, 2007). Depending on the user group, a number of contradictory requirements must be met. Historical researchers, for example, need to be able to compare and contextualize sources (Munster et al., 2015, Brandt, 2012, Wohlfeil, 1986), and to trace the relationship between source and representation (Favro, 2006, Niccolucci and Hermon, 2006). This includes identifying formal patterns, singularities, and discontinuities in architecture and cityscape. This raises a host of questions: How do buildings and cities change over time? In which contexts, such as political or formal developments, does a historical cityscape evolve? What similarities can be found between objects in terms of construction standards and requirements, building codes, regional, temporal or personal tastes and styles?

The research group will address these and many more questions in a specific project on the interdependence between urban development and urban photography.

Creating targeted tools for working with image repositories

An adjacent task will be to perform a systematic survey of the needs of users of image repositories, whose findings will be used to conceptualize technological support options. As historic images, objects and information are increasingly being digitized on a massive scale, more content becomes available for investigation; more cross-analyses are possible; more knowledge is collected, structured and shared (Schuller, 2009). The new scale of research and information retrieval creates many new challenges. Many scholars note that online searching for images and information is “counter-productive” due to the amount of irrelevant data retrieved or their limited technical abilities (Beaudoin and Brady, 2011). Access and efficient data retrieval is inhibited for a variety of reasons. The degree of search expertise is as important as the functionalities and usability of the platform (Kemman et al., 2014). A lot of the existing tools of research programs and applications stem from computer science and do not necessarily meet the needs of humanities scholars (Dudek et al., 2015). Users need efficient search and filter functions, an intuitive software interface and navigation system (Barreau et al., 2014). Appropriate documentation through metadata plays an important role in ensuring sustainability (Bentkowska-Kafel et al., 2012, Maina and Suleman, 2015). In contrast, users expect an intuitive and feasible introduction to the topic and data (Maina and Suleman, 2015) with options to find out more as required. The simplest way to link and contextualize visual information is to use highlighted keywords as hyperlinks in texts and captions. Data interaction and processing tools are also essential for research (Webb and O’Carroll, 2013, Hecht et al., 2015).

Photogrammetric methods of visual knowledge generation

A possible technological basis for creating access to large scale image repositories is the spatial and temporal aggregation of data, in this case historical photographs in a 4D model. The potential of photographic images ranges from pure documentation in archaeology and monument preservation, through image interpretation, for example for damage documentation, to the produc-tion of complex 3D models for archaeological investigations (Buhrer et al., 2001). Geometrical reconstruction from historical photographs is based on photogrammetry. Information from multiple 2D images is used to acquire 2D and 3D object geome-tries and have frequently been applied on historical and measurement images (cf. Wiedemann et al., 2000, Brauer-Burchardt and Voss, 2001, Henze et al., 2009, Siedler et al., 2011). Since some decades, traditional analytical photogrammetry has increasingly been complemented by digital image processing and analysis. The elaborate process of manual image analysis can be largely automated, resulting in large image collections from which geometric in-formation can be generated automatically (Pomaska, 2011). To date, automated photogrammetric methods are generally used primarily to evaluate current, mostly digital images. So far, this has rarely been done for historical images, as it involves specific challenges. Scanned analogue records usually have unknown camera metrics, missing or minimal object information and low radiometric and geometric resolution. Our aim is to develop application-oriented tools for photogrammetric analysis of his-torical photographs, to integrate them into the process of historical image analysis (Fig 1), and to create a spatial relationship to today’s situation.

Figure 1. 3D model based on current photographs and historical photographs (proof-of-concept)

Augmented reality

The prototype 4D model, and the 4D historical photographs, drawings, plans, and information within it, will be made accessible via a location or context related information access as augmented reality (Munster and Niebling, 2016). This technology has gained importance in the last few years and undergone extensive testing (Livingston et al., 2008, Zollner et al., 2010, Walczak et al., 2011, Chang et al., 2015, Chung et al., 2015). Augmented reality describes the

enrichment of the real world through virtual data, which can include 3D models, texts, pictures, films or audio data.

Figure 2. Augmented-reality representation of a cit-yscape (mockup)

Enriching the reality or replacing parts of reality can help to bridge the cognitive gap between our daily life experience of a city-scape and its depiction in historical photographs (Niebling et al., 2008). In the historical context, the viewer is able to interactively capture visual and textual information about objects in their historical spatial reference system (Ridel et al., 2014). Our investigation will focus on the accessibility of historical data: How can interaction with virtual buildings be designed? Which metaphors can be used? How can augmented reality support educational and research settings?

4D browser

Figure 3. 4D-Browser (prototype)

As an alternative path, the 4D model will also be accessible via a 4D browser interface for spatially and temporally located searches in media repositories. An basing application prototype of a research platform for 3D reconstruction projects is in development was developed during a master thesis (Bruschke, 2015), employing approaches, such as semantic data linking and visualization of temporally and spatially arranged information (Gouveia et al., 2015). Since the prototype has focused on individual building complexes, the 4D browser application has to visualize an entire city model, which also changes constantly over time. Moreover, a visual interface is proposed to make additional information accessible, such as the current and original location of the depicted object. Further features intended to support scholarly users of the prospected platform are image rectification tools and overlays combining several pictures from different periods which can shed light on changes in a building. Statistical analyses of photographed objects over time may provide information on a building’s significance. Last but not least, the application should be intuitive to operate for a heterogeneous user group (Warwick, 2012).

Summary

As a result of huge and concerted digitization efforts, extensive digital repositories of historical photographs have been created in the past few decades. This volume of data presents a major challenge to support search, access and information enrich-ment for users. In August 2016, the HistStadt4D research group started exam-ining scientific methodological requirements and intuitive user interfaces for dealing with massive media repositories from a multi-disciplinary perspective.

Bibliography
Deutsche Fotothek (n.d.) [Online]. Available: http://www.deutschefotothek.de/ [Accessed 9.5.2014].

Deutsches Dokumentationszentrum für

Kunstgeschichte - Bildarchiv Foto Marburg (n.d.)

[Online]. Available: http://www.fotomarburg.de/

[Accessed 9.5.2014].

École Polytecnique Fédérale de Lausanne. (2017. The

Venice Time Machine [Online]. Available: http://vtm.epfl.ch/page-109337.html, 11.01.2017

[Accessed].

Barreay, J-B., Gaugne, R., Bernard, Y., Le Cloirec, G., and

Gouranton, V. (2014). Virtual reality tools for the West

Digital Conservatory of Archaeological Heritage.

Proceedings of the 2014 Virtual Reality International

Conference.

Beudoin, J. E., and Brady, E. (2011.) Finding Visual

Information: A Study of Image Resources Used by

Archaeologists, Architects, Art Historians, and Artists.

Art Documentation, 30, 24-36.

Bentowska-Kafel, A., Denard, H., and Baker, D. (2012).

Paradata and Transparency in Virtual Heritage,

Burlington, Ashgate.

Brandt, A. V. (2012). Werkzeug des Historikers, Stuttgart [u. a.], Kohlhammer.

Brassat, W., and Kohle, H. (2003). Methoden-Reader Kunstgeschichte. Texte zur Methodik und Geschichte der Kunstwissenschaft, Köln.

Bräuer-Burchardt, C., and Voss, K. (2001). Facade Reconstruction of Destroyed Buildings Using Historical Photographs. In: Albertz, J. (ed.) Proceedings of the XVIII. International CIPA Symposium 2001, IAPRS, Vol. XXXIV, Part 5/C7, 2001.

Bruschke, J. (2015). DokuVis - Ein Dokumentationssystem fiir Digitale Rekonstruktionen (Master thesis). Master thesis, HTW Dresden.

Bührer, T., Grün, A., Zhang, L., Fraser, C. & Rüther, H.

(2001). Photogrammetric Reconstruction and 3D Visualization of Bet Gorgis, a Rock-hewn Church in Ethiopia. In: Albertz, J. (ed.) Proceedings of the XVIII. International CIPA Symposium 2001, IAPRS, Vol. XXXIV, Part 5/C7, 2001.

Burke, P. (2003). Augenzeugenschaft. Bilder als historische Quellen, Berlin.

Chang, Y.-L., Hou, H.-T., Pan, C.-Y., Sung, Y.-T. and Chang, K.-

E. (2015). Apply an Augmented Reality in a Mobile Guidance to Increase Sense of Place for Heritage Places. Educational Technology & Society, 18, 166-178.

Chung, N., Han, H. and Joun, Y. (2015). Tourists’ intention to visit a destination: The role of augmented reality (AR) application for a heritage site. Computers in Human Behavior, 50, 588-599.

Dudek, I., Blaise, J.-Y., De Luca, L., Bergerot, L. & Renaudid, N. (2015). How was this done? An attempt at formalising and memorising a digital asset's making-of. Digital Heritage, 2, 343-346.

Favro, D., (2006). In the eyes of the beholder. Virtual Reality re-creations and academia. In: Haselberger, L., Humphrey, J. & Abernathy, D. (eds.) Imaging ancient Rome: Documentation, visualization, imagination: Proceedings of the 3rd Williams Symposium on Classical Architecture, Rome, 20.- 23. 5. 2004. Portsmouth: Journal of Roman Archaeology.

Gouveia, J., Branco, F., Rodrigues, A. & Correia, N. (2015). Travelling Through Space and Time in Lisbon’s Religious Buildings. In: Guidi, G., Scopigno, R., TorresJ. C. & Graf, H. (eds.) 2nd International Congress on Digital Heritage 2015. Granada.

Hecht, R., Meinel, G. & Buchroithner, M. F. (2015). Automatic identification of building types based on topographic databases - A comparison of different data sources. International Journal of Cartography, 1, 18-31.

Henze, F., Lehmann, H. & Bruschke, B. (2009). Nutzung historischer Plane und Bilder für die Stadtforschungen in Baalbek / Libanon. Photogrammetrie - Fernerkundung -Geoinformation, 3, 221-234.

Kemman, M., Kleppe, M. & Scaglolia, S. (2014). Just Google It. Digital Research Practices of Humanities Scholars. Proceedings of the Digital Humanities Congress 2012. Studies in the Digital Humanities. Sheffield: HRI Online.

Kwastek, K. (2014). Vom Bild zum Bild. Digital Humanities jenseits des Texts (Keynote). 1. Jahrestagung der Digital Humanities im deutschsprachigen Raum (DHd 2014). Passau.

Livingston, M. A., Bimber, O. & Saito, H. (2008). Proceedings of the 7th IEEE International Symposium on Mixed and Augmented Reality. Cambridge, UK, Piscataway, IEEE Xplore.

Maina, J. K. & Suleman, H. (2015). Enhancing Digital Heritage Archives Using Gamified Annotations. In: Allen, B. R., Hunter, J. & Zeng, L. M. (eds.) Digital Libraries: Providing Quality Information: 17th International Conference on Asia-Pacific Digital Libraries, ICADL 2015, Seoul, Korea, December 9-12, 2015. Proceedings. Cham: Springer International Publishing.

Münster, S. (2011). Entstehungs- und

Verwendungskontexte von 3D-CAD-Modellen in den Geschichtswissenschaften. In: Meissner, K. & Engelien, M. (eds.) Virtual Enterprises, Communities & Social Networks. Dresden: TUDpress.

Münster, S.., Jahn, P.-H. & Wacker, M. (2015). Von Plan- und Bildquellen zum virtuellen Gebaudemodell. Zur Bedeutung der Bildlichkeit fiir die digitale 3D-Rekonstruktion historischer Architektur. In: Ammon, S. & Hinterwalder, I. (eds.) Bildlichkeit im Zeitalter der Modellierung. Operative Artefakte in Entwurfsprozessen der Architektur und des Ingenieurwesens. Munchen: Wilhelm Fink Verlag.

Münster, S. & Niebling, F. (2016.) HistStadt4D -Multimodale Zugange zu historischen Bildrepositorien zur Unterstützung stadt- und baugeschichtlicher Forschung und Vermittlung. Digital Humanities im deutschsprachigen Raum (DHd) 2016. Duisburg: nisaba verlag.

Niccolucci, F. & Hermon, S. (2006). A Fuzzy Logic Approach to Reliability in Archaeological Virtual Reconstruction.

In: Niccolucci, F. & Hermon S. (eds.) Beyond the Artifact. Digital Interpretation of the Past. Budapest.

Niebling, F., Griesser, R. T. & Woessner, U. (2008). Using Augmented Reality and Interactive Simulations to Realize Hybrid Prototypes. Advances in Visual Computing, 4th International Symposium, ISVC 2008 (Proceedings, Part I). Las Vegas, NV.

Paul, G. (2006). Von der Historischen Bildkunde zur Visual History. Visual History. Ein Studienbuch. Gottingen.

Perez-Gomez, A. & Pelletier, L. (1997). Architectural Representation and the Perspective Hinge, Cambridge, London, University Press.

Pomaska, G. (2011). Zur Dokumentation und

3D-Modellierung von Denkmalen mit digitalen fotografischen Verfahren. In: Heine, K., Rheidt, K., Henze, F. & Riedel, A. (eds.) Von Handaufmaß bis High Tech III -3D in der historischen Bauforschung. Mainz: Verlag Philipp von Zabern.

Ridel, B., Reuter, P., LaViole, J., Mellado, N., Couture, N. & Granier, X. (2014). The Revealing Flashlight: Interactive Spatial Augmented Reality for Detail Exploration of Cultural Heritage Artifacts. J. Comput. Cult. Herit., 7, 1-18.

Schuller, G. (2009). Designing universal knowledge, Baden, Lars Muller Publishers.

Siedler, G., Sacher, G. & Vetter, S. (2011.

Photogrammetrische Auswertung historischer Fotografien am Potsdamer Stadtschloss. In: Heine, K., Rheidt, K., Henze, F. & Riedel, A. (eds.) Von Handaufmaß bis High Tech III - 3D in der historischen Bauforschung. Mainz: Verlag Philipp von Zabern.

Verstegen, U. (2007). Vom Mehrwert digitaler Simulationen dreidimensionaler Bauten und Objekte in der architekturgeschichtlichen Forschung und Lehre. Vortrag am 16.3.2007. XXIX. Deutscher

Kunsthistorikertag, 2007 Regensburg.

Walczak, K., Cellary, W. & Prinke, A. (2011). Interactive Presentation of Archaeological Objects Using Virtual and Augmented Reality. In: Jerem, E., Redo, F. & Szevereni, V. (eds.) On the Road to Reconstructing the Past. Proceedings of the 36th International Conference on Computer Applications and Quantitative Methods in Archaeology (CAA). Budapest: Archaeolingua.

Warwick, C. (2012). Studying users in digital humanities. In: Warwick, C., Terras, M. & Nyhan, J. (eds.) Digital Humanities in Practice. London: Facet Publishing,.

Webb, S. & O'Carroll, A. (2013). Digital Heritage Tools in Ireland - a Review. Papers of Cultural Heritage, Creative

Tools and Archives, 26.-27.06.2013, National Museum of Denmark, Copenhagen,

Wiedemann, A., Hemmleb, M. & Albertz, J. (2000.) Reconstruction of historical buildings based on images from the Meydenbauer archives. International Archives of Photogrammetry and Remote Sensing, XXXIII, 887893.

Wohlfeil, R. (1986). Das Bild als Geschichtsquelle. Historische Zeitschrift, 243, 91-100.

Zöllner, M., Becker, M. & Keil, J. (2010). Snapshot Augmented Reality - Augmented Photography. In: Artusi, A., Joly-Parvex, M., Lucet, G., Ribes, A. & Pitzalis, D. (eds.) 11th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST 2010). Paris: Eurograph