Fit for Purpose: Issues Surrounding the Use of Digital Resources in Research and Teaching

Humanities disciplines have a large and varied body of digital resources on which to draw for teaching and research purposes which include scholarly editions, on-line dictionaries and journals, collections of digital texts and images, large Internet gateways and numerous individual and course Web pages. In spite of the recent quick growth of digital resources, there are no guidelines published, as far as I am aware, which assist the academic in assessing the quality of a digital resource for actual use in teaching or research. This is not surprising. First, the answer to whether a resource is fit for a purpose will almost always be, 'it depends'. Not only does the answer depend on the general purpose envisaged, whether for teaching or research, it will also inevitably depend on the precise needs of the individual asking the question, for there is a whole spectrum of approaches to the subject even within a single discipline. Secondly, it is not surprising that no set of criteria exists for determining the quality of a digital resource when so little published criteria exist for assessing the quality of academic research in general. It is a contentious issue, as the United Kingdom academic community who have been subject to the Research Assessment Exercise will confirm. But the issue also comes to the fore within the evaluation process for tenure, the acceptance of publications by publishers and editorial boards, and the success or otherwise of research funding. Underpinning all of these is some notion of peer-review and the refereeing process which remains crucial in the assessment of research publications.

How appropriate is the application of the peer-review methods to the assessment of digital resources? In summary one might argue that digital resources should not be treated any differently from other resources. The peer-review process is as appropriate for determining their 'usefulness', as it is for effecting their development, publication and, hopefully, the academic rewards structure. The assessment of digital resources, however, whilst always requiring expert knowledge of the subject area, also requires an understanding of the underlying technology. Acknowledged experts in manuscript studies, for example, simply may not appreciate the potential scholarly contribution of an electronic facsimile, if the digital medium itself is significantly more alien to them, than the publication of another printed facsimile. The recognition that subject experts must understand the potential of the technology employed, in order to assess the quality of a resource, was apparent in the recent study undertaken by the Arts & Humanities Data Service into the requirements of academics for the scholarly use of digital resources (see http://ahds.ac.uk/public/uneeds/un0.html). The Oxford Text Archive reported not only that their academic users perceived as obstacles to the use of digital resources the technical ability required to use certain resources and the corresponding lack of training available, but also the current proliferation of resources which by-pass the benefits of academic review.

Academic practitioners who do have a familiarity with current and emerging technologies, however, have come to expect more from a digital resource than can be delivered on paper. The electronic edition of a medieval text is no longer a novelty. For both research and teaching purposes there is almost an accepted expectation that a critical edition in digital form will comprise not only the full texts, but also the high quality digital facsimiles of all the surviving witnesses. Moreover the witnesses are expected to be encoded for advanced searching and linked to supplementary materials such as glossaries and textual notes. In such cases it is less a single resource than an entire scholarly environment that projects are expected to provide. The editions are expected to be easy to use and transparent even for a student inexperienced in both their academic and technical aspects. As is well known digital resources which strive to meet such expectations are often expensive, very time consuming undertakings, requiring unremitting devotion and extremely hard work from the teams which create them. These difficulties are acknowledged by some members of the academic community in that they make a positive evaluation of digital resources as suitable for academic use, in spite of, for example, the lack of high quality digital photography (accepting that it is expensive and that permissions are difficult to obtain), or (accepting the need to work in the situation of ever changing and developing technology) in spite of occasional technological failures, the lack of compatibility with all the existing platforms, or their slowness even on the most current computers. Other members of academic community are however less forgiving of these conditions 'outside the editor's control', and lose confidence in digital resources.

Another difficulty well-known to any 'insider' is that the huge quantities of data which underlie electronic resources often exist in a form which makes them especially difficult to proofread. In these situations the proofreading and checking done by the project team under the increasing pressure of deadlines never seems sufficient and may have to stop before complete satisfaction is achieved. Again the arguments for making the results of work available in spite of certain imperfections, and the dangers which may result from this are not easy to balance. One particular danger is that as a result of a combination of a large body of complex data in a digital resource with a lack of technical expertise on the users' part, a resource might be used or recommended for scholarly use for some time before its faults become apparent. Any 'forgiving' attitude on the part of the academic community, which itself may benefit from the early publication of a cutting-edge resource still imperfect in some aspects, should require an honest assessment of the resource by its creators and an open statement of its weaknesses. The promise of easy updating which comes with digital technology justly encourages a 'forgiving' attitude, but should not be used as a justification for the publication of poor quality work.

The input of scholars with different backgrounds is required for the evaluation of digital resources within collections and gateways. Ultimately, the inclusion of a resource within a peer-reviewed gateway or a 'published' digital collection should have the same effect upon its use and the reward of its creators as is associated with current publishing activities.