A Case Study of an Evaluation Questionnaire Concerning the Integration of a Hypermedia Project in University Courses

Summary

The presentation will develop the results of a questionnaire survey following the introduction of hypermedia products in cultural history and humanities computing courses. This experiment concerns the use of a hypermedia CD-ROM on Georgian Cities (developed by the Research Centre 'Cultures Anglophones et Technologies de l' Information' CATI at the Université de Paris-Sorbonne), a prototype of which was demonstrated in a paper at ACH-ALLC 1999; in the academic year 1999-2000 it is used in teaching in four universities, Paris-Sorbonne, Nanterre, Lille and Mulhouse, both in classes on eighteenth-century cultural history and on humanities computing. Concurrently, a website was created, meant to serve as supporting material for the courses, containing chapters presenting the CD-ROM, sections on documentation methods and gateways to relevant websites on the 18th century (in particular 'virtual cities' sites developed by the Universities of Bath and Missouri), on literary resources, and on architectural history. This was part of a strategy to develop the teaching of documentation methods and electronic resources in humanities courses, with which the research centre was entrusted.

Presentation

The presentation will consist of a demonstration of the products based on the analysis of the questionnaires, highlighting the sections which were shown by this survey to be significant in terms of learner use of media. Illustrative material such as syllabus outlines, students' worksheets and documents used concurrently will be available, as well as samples of website pages.

The survey

It was based on four questionnaires, two for teachers and two for students, in each case one pre-program and one post-program. The questionnaires were worked out on the models recommended by the Oxford CTI (Porter) and by the Open University's Programme on Learner Use of Media (PLUM). The CTI questionnaire was used especially for its structure relating the general background of humanities computing courses and the details of the current experiment, and for its format. The PLUM recommendations which were particularly relevant were those on alternative conditions of use in the instructional context, and on the interaction between interface and contents, assessing improvements both in approach and in knowledge. The points emphasized in the present questionnaires focused on two main types of questions: 1) the integration of IT resources in the syllabus (Deegan) - apart from the present CD-ROM, and also including it 2) interface/navigation procedures for the present CD-ROM, and their relation with the contents, focusing on the potential of multimedia presentation for the development of interdisciplinary approaches (Zuern). The survey also included monitoring reports by the teachers, where they recorded the stages in the students' use of the CD-ROM, and their methods for resorting to other materials used in conjunction (for instance comparison with websites or paper documents, note-taking).

The results
areas selected by students: two main types of navigation paths emerged.
A large number of students focused on general chapters on demography or education; they had a particular project as a purpose and were using this program as a resource in which to look for explanations on the topic. A graduate student doing research on townscape painting in the 18th century consulted the CD-ROM to find documents on the subject and new approaches to it, taking notes; she was also interested in finding methods of presentation for visual questions which could hardly be conveyed in other media (modelling of changing shadow effects for instance).
Others selected topographical navigation in one of the cities. They were seeking multidisciplinary contextualisation to their subject studies (for instance in literature - e.g. Smollett - or in social history). Their comments were that it brought to life subjects which so far they had not visualized so clearly.
renewal of approach
IT / subject: This experiment was initially integrated into the current syllabus and tasks, to which it was meant to provide an extra dimension. Its more innovative benefits were that it opened vistas on new approaches to the subject, thus encouraging critical thinking about the relative interest of various methodologies (Talarico). It helped students to formulate such questions on the definition of subject approaches and make a more critical use of IT.
students' role: Their responses differed according to their previous experience of IT. The classes caused students' interest in the teachers' preparation work for the documents used in class (an attitude which is not so explicit for traditional methods of teaching). In the sections of the courses where the teachers (who do the authoring work) demonstrated how to construct a program like the one they had just seen as a finished product, they were interested in being taken behind the scenes, especially if they had previous experience of authoring programs themselves (for instance doing a personal website); it enabled them to compare their practice with the teacher's. In that case, having themselves experienced the difficulties of scripting, they were surprised to see that it worked first time (which seemed normal to those who have no experience of computers). It encourages students to relate their own extracurricular interests to their studies. Those who so far had used IT started realising that it was not beyond reach since they could see it done under their eyes. They were also asked to do follow-up work, finding relevant sites on the Internet, which helped them to practise navigation and to formulate queries (use of keywords, Boolean operators, interpretation of the results in terms of 'noise' and 'silence' where necessary). Generally speaking, students appreciated questions about possible improvements to the CD-ROM, and listed corrections they suggested, which gave them an active role.
conclusions for the teaching methods
It helps teachers identify issues in the pedagogical use of IT. The faculty members involved in the project took part in it because it enabled them to add a variety of approaches to the subject. They could compare the guided use of such resources - exploration according to a path selected by the teacher - and the individual use by students. In addition, other colleagues who taught courses on related subjects - e.g. a literature course on Smollett and Jane Austen - asked for a demonstration to be given to their class, both as background information on Georgian Bath and for the extracts of film adaptations of the novels. It was a means to introduce such material to faculty members who had not used it so far.
The analysis of the survey
A detailed analysis of the survey will show the results sorted according to several criteria, e.g. level of studies, or subject. It will also correlate the answers according to previous experience of IT in the curriculum.

References

Deegan, Marilyn (2000). 'From innovation to integration.' Villes visite virtuelle. PUPS, Paris. 7-19.
PLUM <http://iet.open.ac.uk/Plum/evaluation/contents.html>
Porter, Sarah, and Lisa McRory (1998). 'Digital Text in Humanities Teaching.' In Lou Burnard, Marilyn Deegan and Harold Short (eds) The Digital Demotic: A Selection of Papers from Digital Ressources in the Humanities 1997. King's College, Office for Humanities Communication, London.
Talarico, Kathryn Marie (1999). 'Cyberspace without Tears: Fundamental Approaches to the Uses of Technology in the Classroom.' Literary and Linguistic Computing 14.2 (June 1999)199-210.
Zuern, John (1999). 'Timelines OnLine: Hypermedia and Information Architecture in the Representation of Intellectual History.' Paper at ACH-ALLC, University of Virginia, 12 June 1999.