More Than Just CG: Storytelling And Mixed Techniques In An Educational Short Movie

Introduction

“Il piccolo Masaccio e le Terre Nuove” is a short animated Computer Graphics video explaining the origins and the history of San Giovanni Valdarno, a city of foundation (Cardini, 2009). The video was created for the
“Museo delle Terre Nuove”, which offers an overview of the phenomenon of the New Towns born in Tuscany, Italy and Europe during the Middle Ages. The main character of the video is a young version of the painter Masaccio (Tommaso), born in 1401 in Castel San Giovanni, the earlier name of San Giovanni Valdarno. Tommaso roams the city during a siege and, on top of the tower of Arnolfo’s palace, meets the Vicar, ruler of the town on behalf of the city of Florence, who explains him the rational rules underlying the creation of San Giovanni (Bertocci, 2003); (Bianchini, 2003); (Puma, 2003) (Fig. 4).

In order to communicate to the visitors a unique experience, combining education and entertainment, the video exploitsof a series of very different techniques, such as:

live shots, taken also by drone;
Computer Graphics, with photogrammetric, procedural and geometric 3D modeling, crowd (Fig. 3), cloth and particle simulations and a digital library for the vegetation of the time (Fig. 6);
watercolours and 2D drawings, made also with a tablet device (Fig. 1);
digital videos taken from Google Earth.

Figure 1 - Watercolours painted for the video
Besides
CityEngine, used for the procedural modelling of San Giovanni (Fig. 2), and
PhotoScan, used for some materials and minor details of the buildings, the work was realised in
Blenderin an Open Source pipeline.

Figure 2 - Procedural reconstruction of San Giovanni

Fig 3 - Simulating a crowd of soldiers

Figure 4 - A view of the city

This contribution presents an overview of the solutions adopted in the development of a CG educational video in order to both attract more visitors to the museum and inspire them with the desire to deepen its topics.

Computer Graphics and videos for Museum communication
The use of documentaries to present cultural contents in museums is not new. A short video allows a fairly rapid rotation of the spectators and is generally well compatible with pre-existing museum displays. Over time, real-life documentaries have been joined by 2D educational cartoons and, later, by Computer Graphics videos. The latter, however demanding, allows us to reconstruct, re-contextualize, explain and show different physical perspectives and make landscapes and events of the past much better understandable (Forte, 2017); (Ferdani and Pescarin, 2012); (Märker et al., 2012), enabling also the insertion of particular graphic elements into reality, as in the application of the Tate Britain Museum (Shi, 2016). Usually, Computer Graphics is used to show architectural reconstructions (Gabellone et al., 2017). However, the emotional narration is gaining ground, increasingly introducing storytelling (Perry et al., 2017)(Staus and Falk, 2017), also for interactive applications, such as
CrazyTalk, created for the

Thyssen-Bornemitzsa Museum.

Figure 5 - Tommaso on top of Arnolfo’s tower
Since they are cheaper, actors are preferred over animated characters. In virtual sets (Pietroni, 2017) or alongside CG integrations, to show landscapes or reconstructions of the past (as in the video at
Roman Army Museum, Vindolanda). Storytelling with 3D animated characters is still limited because of its high costs and challenges (Guidazzoli et al., 2011), even if, thanks to tools such as
Cookie FlexRig it is possible to create them quicker than before (Guidazzoli et al., 2016). As stated by Vosinakis (Vosinakis, 2017), a key requirement for digital characters is credibility, that is more a “coherence of this character within the world it is situated in”, than a high degree of realism in its realisation. Believability involves the character’s behaviour as expressed by gaze, facial expression, gesture and posture, making the creation of a character even more complex and time consuming (Fig. 8). However, the use of characters inside a narration offers a significant contribution for capturing the audience’s attention and entertain it. Characters, atmospheres, dialogues and history, can all transform a passive activity into something more engaging (Gabellone et al., 2016), trigger the visitors’ interest, empathy and imagination, leading them towards a successfully entertaining experience (Roussou, 2008). Vosinakis remembers us to “...focus on the expressive side of the digital characters, even by incorporating non realistic elements, rather than striving for accuracy and realism” (Vosinakis, 2017). However, when working at a communicative product for a museum, it is very important to keep a good level of philological accuracy.

All these elements were central while developing “Il piccolo Masaccio”, implying a great commitment to philologization, for example in choosing and modeling clothes, and, at the same time, the adoption of something peculiar and cartoonish for attracting the viewer, such as the large eyes of Tommaso or its hair, stained with painting (Fig. 5).

Figure 6 - Distribution of the vegetation on the map

Directing a CG educational video
“Il piccolo Masaccio” is a peculiar educational animated movie, both for the multiple techniques and for the different visual solutions adopted. The video explains how the “New Towns” of the fourteenth century, such as San Giovanni Valdarno, were born and on which principles and rules of political and architectural design were based. However, “teaching” through an animated video can risk of not reaching the goal because an excess of didacticism reduces its appeal (especially with a young audience). A varied visual poetic was used, therefore, to give more rhythm to the story and avoid a boredom effect. For example, the cold and precise lines of the digital houses were totally replaced by irregular and sweeter lines. Even the colours of the façades of the houses were made softer and warmer, inspired by the same frescoes by Masaccio and his contemporaries, such as Masolino (Spike, 1995). Surfaces and colours are never clean, but dirty, with darker patches to suggest the idea of real walls, scratched sometimes or affected by damp and mould (Fig. 7).

Figure 7 - Creating a credible town of the Renaissance

To give more emphasis to the film and to further soften the 3D model of the city, we resorted to a very blue sky with white clouds, almost wisps of whipped cream, reminiscent of Hayao Miyazaki’s skys (Arnaldi, 2014). Camera movements were also devised in order to be more independent from a classic navigation of a 3D model. In addition to 3D computer graphics, it was deemed appropriate to use 2D graphics (Keech et al., 2017). This choice enabled the explanation of geometric principles in a simpler way. Furthermore, we used hand-made drawings with ink and watercolours to lighten the narration and enrich the film with visual suggestions, which culminate when Tommaso, inspired by the Vicar’s narration, draws in an infinite and imaginary space a series of ideal cities. Live shots were adopted in order to link past and present and to highlight the continuity of history, where everything we now see is the result of what was conceived, designed and built in the past.

Figure 8 - Study of characters’ expressions

Conclusions
Creating a 3D world requires a significant amount of time and resources, especially compared to a real-time shooting or, even, a 2D movie, which could lead to similar results in terms of communication. The 9 minute video “Il piccolo Masaccio e le Terre Nuove” required the effort of a team of six person for one full year; some small contributions of high-school students; the help of 2 video-makers, for live footages, and of a professional animator. In the end the repository hosted 15,3 GB of Blender files and textures and 11 GB of high resolution frames. The rendering is in Cycles, the Blender unbiased rendering engine, and for the low resolution version and its work in progress 72,000 core hours were used; 65,000 core hours more for the high-res version. Renderings were launched on the Cineca Blender Render Farm, scripted
ad hoc for this project with the new SLURM scheduler on the supercomputer Galileo
360 Compute nodes (Intel Broadwell), each with 2*18-core Intel Xeon E5-2697 v4 @ 2.30GHz - 128 GB RAM. The consistent effort became in time a co-production, with Cineca supporting the economic possibilities of the Museum. However, Computer Graphics also provided a higher level of philological accuracy, thanks to the multiplicity of solutions available, such as procedural modelling tools or material or vegetation libraries, plus a better support to the director’s creativity. Besides, once the 3D assets have been built, they are reusable in further applications. But, above all, CG and storytelling can be a powerful attractor towards a young audience, used to high quality productions.

The movie was premièred at the museum during the National Families at the Museum Day on October 2018. A questionnaire was submitted to the younger audience and it collected praises from everybody. Asked to draw the favourite character from the video, the kids privileged Tommaso, but also the vicar and Tommaso’s mother got their share of attention.
From then on, the repeated submission of questionnaires to the audience will help in fully understand the capacity of “Il piccolo Masaccio e le Terre Nuove” in transmitting the desired contents and attracting more visitors to the museum.

Bibliography

Arnaldi, V. (2014).
Hayao Miyazaki: il mondo incantato. Roma: Ultra.

Bertocci, S. (2003). Castel San Giovanni ai tempi di Arnolfo. In Bartoli, M. and Bertocci, S. (eds),
Città e Architettura. Le Matrici Di Arnolfo. Firenze: Edifir, pp. 81–93.

Bianchini, L. (2003). La fortificazione delle Terre Nuove: le mura. In Bartoli, M. and Bertocci, S. (eds),
Città e Architettura. Le Matrici Di Arnolfo. Firenze: Edifir, pp. 95–106.

Cardini, F. (2009).
Dalla ‘Terra Nuova’ Alla Città Contemporanea: Storia Di San Giovanni Valdarno. (Arte). Ospedaletto (Pisa) : [San Giovanni Valdarno, Italy]: Pacini ; Comune di San Giovanni Valdarno.

Ferdani, D. and Pescarin, S. (2012). Dal GIS alla ricostruzione 3D del paesaggio antico.
Atti XIII Conferenza Italiana Utenti ESRI 2012 http://www.esriitalia.it/images/Atti_13aConfer/Sessioni_parallele_pdf/Ricostruzioni/05_Ricostruzioni.pdf.

Forte, M. (2017). Il paesaggio virtuale di Regium Lepidi. In Cantoni, G. and Capurso, A. (eds),
On the Road, via Emilia 187 a.C. - 2017, Catalogo Della Mostra. Parma: Grafiche Step.

Gabellone, F., Ferrari, I., Giuri, F. and Chiffi, M. (2016). What communication for museums? Experiences and reflections in a virtualization project for the Museo Egizio in Turin.
Proceedings of the 20th International Conference on Cultural Heritage and New Technologies 2015 (CHNT 20, 2015). Wien: Museen der Stadt Wien – Stadtarchäologie https://www.chnt.at/wp-content/uploads/eBook_CHNT20_Gabellone_etal_2015.pdf.

Gabellone, F., Ferrari, I., Giuri, F. and Chiffi, M. (2017). Image-Based techniques for the virtualization of Egyptian contexts.
3rd IMEKO International Conference on Metrology for Archaeology and Cultural Heritage. Lecce, pp. 601–06.

Guidazzoli, A., Liguori, M. C., Calori, L., Ponti, F. D., Diamanti, T., Imboden, S., Mauri, A., Negri, A., Cohen, G. B. and Pescarin, S. (2011). Apa the Etruscan and 2700 years of 3D Bologna history.
SIGGRAPH Asia 2011 Posters on - SA ’11. Hong Kong, China: ACM Press, p. 1 doi:10.1145/2073304.2073307. http://dl.acm.org/citation.cfm?doid=2073304.2073307 (accessed 4 March 2019).

Guidazzoli, A., Liguori, M. C., Imboden, S., De Luca, D., Bellavia, G. and Verri, L. (2016). Ati the Etruscan: a transmedia CG character for educational storytelling.
SIGGRAPH ASIA 2016 Symposium on Education on - SA ’16. Macau: ACM Press, pp. 1–8 doi:10.1145/2993352.2993353.

Keech, K., Bibb, R., Whited, B. and Achorn, B. (2017). The role of hand-drawn animation in Disney’s
Moana.
ACM SIGGRAPH 2017 Talks on - SIGGRAPH ’17. Los Angeles, California: ACM Press, pp. 1–2 doi:10.1145/3084363.3085079. http://dl.acm.org/citation.cfm?doid=3084363.3085079 (accessed 4 March 2019). http://dl.acm.org/citation.cfm?doid=2993352.2993353 (accessed 4 March 2019).

Märker, M., Vogel, S. and Seiler, F. (2012). Reconstructing the ancient cultural landscape around pompeii in 2D and 3D. Southampton (U.K.).

Perry, S., Roussou, M., Economou, M., Young, H. and Pujol, L. (2017). Moving beyond the virtual museum: Engaging visitors emotionally.
2017 23rd International Conference on Virtual
System & Multimedia (VSMM). Dublin: IEEE, pp. 1–8 doi:10.1109/VSMM.2017.8346276. https://ieeexplore.ieee.org/document/8346276/ (accessed 4 March 2019).

Pietroni, E. (2017). Virtual Museums for Landscape Valorization and Communication.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,
XLII-
2/W5: 575–82 doi:10.5194/isprs-archives-XLII-2-W5-575-2017.

Puma, P. (2003). Geometria, misura e facies urbana. In Bartoli, M. and Bertocci, S. (eds),
Città e Architettura. Le Matrici Di Arnolfo. Firenze: Edifir, pp. 107–09.

Roussou, M. (2008). The components of engagement in virtual heritage environments. In Kalay, Y., Kvan, T. and Affleck, J. (eds),
New Heritage: New Media and Cultural Heritage. pp. 225–41.

Shi, D. (2016). Turn the Tate Museum into an Insane Funhouse with this Animation. A digital walk-through of the Tate Britain features VR-like animations in a setting full of surprises.
Vice https://www.vice.com/en_au/article/yp5yg7/tate-museum-playful-cg-animation (accessed 4 March 2018).

Spike, J. T. (1995).
Masaccio. Milano: Fabbri.

Staus, N. L. and Falk, J. H. (2017). The Role of Emotion in Informal Science Learning: Testing an Exploratory Model: The Role of Emotion in Informal Science Learning.
Mind, Brain, and Education,
11(2): 45–53 doi:10.1111/mbe.12139.

Vosinakis, S. (2017). Digital Characters in Cultural Heritage Applications:.
International Journal of Computational Methods in Heritage Science,
1(2): 1–20 doi:10.4018/IJCMHS.2017070101.