The Master Builders: LAIRAH Research on Good Practice in the Construction of Digital Humanities Projects

Abstract:
This paper describes the results of research carried out
during the LAIRAH (Log analysis of Internet Resources
in the Arts and Humanities) project (<http://www.ucl.a
c.uk/slais/circah/lairah/>) which is based at
UCL’s School of Library Archive and Information Studies. It
was a fifteen month study (reporting in October 2006) to
discover what influences the long-term sustainability and use
of digital resources in the humanities through the analysis and
evaluation of real-time use.
At Digital Humanities 2006 we reported on the early stages of
the project, in which we carried out deep log analysis of the
AHDS and Humbul portals to determine the level of use of
digital resources. (Warwick et al. 2006) This proposal will
discuss the results of the final phase of the research in which
we examined digital resources from the point of view of those
who designed and built them. We aimed to discover whether there were common characteristics and elements of good
practice linking resources that are well- used.
Numerous studies have been carried out into the information
needs and information seeking practices of humanities scholars
(Barrett, (2005) Talja and Maula (2003), Herman (2001) and
British Academy, (2005)). However, our research is original
because it surveys the practices of those who produce digital
humanities resources. We also based the selection of our
projects on deep log analysis: a quantitative technique which
has not previously been applied to digital humanities resources
to ascertain real usage levels of online digital resources.
Method:
We selected a sample of twenty one publicly funded
projects with varying levels of use, covering different
subject disciplines, to be studied in greater depth. We classified
projects as well-used if the server log data from the Arts and
Humanities Data Service (AHDS) and Humbul portals showed
that they had been repeatedly and frequently accessed by a
variety of users. We also mounted a questionnaire on these sites
and asked which digital resources respondents found most
useful. Although most nominated information resources, such
as libraries, archives and reference collections for example the
eDNB, three UK publicly funded research resources were
mentioned, and thus we added them to the study. We also asked
representatives of each AHDS centre to specify which resources
in their collections they believed were most used. In the case
of Sheffield University the logs showed that a large number of
digital projects accessed were based at the Humanities Research
Institute. We therefore conducted interviews about the HRI and
its role in fostering the creation of digital humanities resources.
The selected projects were studied in detail, including any
documentation and reports that could be found on the project’s
website. We also interviewed a representative of the project,
either the principal investigator or a research assistant.
Results:
Institutional context:
The majority of projects that we interviewed had been well
supported in technical terms, and this had undoubtedly aided
the success of the project, especially where it was associated
with a centre of digital humanities excellence such as the Centre
for Computing in the Humanities at Kings College London or
the HRI at Sheffield. Critical mass aided the spread of good
practice in the construction and use of digital resources in the
humanities. Where a university valued such activities highly
they tended to proliferate. More junior members of staff were
inspired to undertake digital humanities research by the success
of senior colleagues and early adopters respected for their
traditional and digital research. Unfortunately such critical mass
is relatively rare in UK universities and some PIs reported that
their digital resource was not understood or valued by their
departments, and thus their success had not lead to further
digital research.
Staffing:
PIs also stressed how vital it had been to recruit the ideal RAs.
These were however relatively difficult to find, as they had to
have both disciplinary research expertise and good knowledge
of digital techniques. Most RAs therefore required training,
which many PIs often found lacking or of poor quality. A
further frustration was the difficulty of finding funding to
continue research, this meant that an expert RA might leave,
necessitating further training of a new employee if the project
was granted future funding.
Dissemination:
The strongest correlation between well-used projects and a
specific activity was in the area of dissemination. In all the
projects studied, staff had made determined efforts to
disseminate information as widely as possible. This was a new
challenge for many humanities academics, who were more used
to writing books, marketed by their publishers. This might
include giving papers at seminars and conferences both within
the subject community and the digital humanities domain;
sending out printed material; running workshops, and in the
most unusual instance, the production of a tea-towel!
User contact:
Very few projects maintained contact with their users or
undertook any organised user testing, and many did not have
a clear idea how popular the resource was or what users were
doing with it. However, one of the few projects that had been
obliged to undertake user surveys by its funders was very
well-used, and its PI had been delighted at the unexpected
amount and range of its use. Another project came to the belated
realisation that if it had consulted users the process of designing
the resource would have been simpler and less demanding.
Documentation:
Few of the projects kept organised documentation, with the
exception of those in archaeology, linguistics and archival
studies, where such a practice is the norm in all research. Most
projects had kept only fragmentary, internal documents, many
of which would not be comprehensible to someone from outside. Documentation could also be difficult to access, with
only a small minority of projects making this information
available from its website. This is an important omission since
documentation aids reuse of resources, and also provides vital
contextual information amount its contents and the rationale
for its construction that users need to reassure them about the
quality of the resource for academic research.
Sustainability:
Another area of concern was the issue of sustainability.
Although the resources were offered for deposit with the AHDS,
few PIs were aware that to remain usable, both the web interface
and the contents of the resource would require regular updating
and maintenance, since users tend to distrust a web page that
looks outdated. Yet in most cases no resources were available
to perform such maintenance, and we learnt of one ten year old
resource whose functionality had already been significantly
degraded as a result.
Conclusion and recommendations
Well-used projects do therefore share common features
that predispose them to success. The effect of
institutional and disciplinary culture in the construction of
digital humanities projects was significant. We found that
critical mass was vital, as was prestige within a university or
the acceptance of digital methods in a subject. The importance
of good project staff and the availability of technical support
also proved vital. If a project as to be well-used it was also
essential that information about it should be disseminated as
widely as possible.
Even amongst well-used projects, however we found areas that
might be improved, these included organised user testing, the
provision of and easy access to documentation and the lack of
updating and maintenance of many resources.
Recommendations:
Documentation:
• Projects should keep documentation and make it available
from the project web site, making clear the extent,
provenance and selection methods of materials for the
resource.
• Funding bodies might consider making documentation a
compulsory deliverable of a funded project.
• Discussions could be held between relevant stakeholders
and the funding bodies, with the aim of producing an agreed
documentation template. This should specify what should
be documented and to what level of detail.
Users:
• Projects should have a clear idea of whom the expected
users might be; consult them as soon as possible and
maintain contact through the project via a dedicated email
list , website feedback or other appropriate method
• They should carry out formal user surveys, software and
interface tests and integrate the results into project design.
• Applicants for funding should show that they have consulted
documentation of other relevant projects and discuss what
they have learnt from it in their case for support. The results
of such contact could then be included in the final report as
a condition of satisfactory progress.
Management:
• Projects should have access to good technical support,
ideally from a centre of excellence in digital humanities.
• Projects should recruit staff who have both subject expertise
and knowledge of digital humanities techniques, then train
them in other specialist techniques as necessary.
• Funding bodies might consider requiring universities to
offer more training for graduate students and RAs in digital
humanities techniques.
Sustainability:
• Ideally projects should maintain and actively update the
interface, content and functionality of the resource, and not
simply archive it with a data archive such as the AHDS.
However this is dependent on a funding model which makes
this possible.
Dissemination:
• Disseminate information about itself widely, both within
its own subject domain and in digital humanities.
• Information should be disseminated widely about the
reasons for user testing and its benefits, for example via
AHRC/AHDS workshops. Projects should be encouraged
to collaborate with experts on user behaviour. Acknowledgements:
This project was funded by the Arts and Humanities
Research Council ICT Strategy Scheme. We would also
like to thank all of out interviewees for agreeing to talk to us.
Bibliography
Barrett, A. "The Information Seeking Habits of Graduate
Student Researchers in the Humanities." The Journal of
Academic Librarianship 31.4 (2005): 324-331.
British Academy. E-resources for Research in the Humanities
and Social Sciences - A British Academy Policy Review section
3.5 . 2005. <http://www.britac.ac.uk/reports/e
resources/report/sect3.html#part5>
Herman, E. "End-users in Academia: Meeting the Information
Needs of University Researchers in an Electronic Age Part 2
Innovative Information-accessing Opportunities and the
Researcher: User Acceptance of IT-based Information
Resources in Academia." Aslib Proceedings. 2001. 431-457.
Talja, S., and H. Maula. "Reasons for the Use and Non-use of
Electronic Journals and Databases - A Domain Analytic Study
in Four Scholarly Disciplines." Journal of Documentation 59.6
(2003): 673-691.
Warwick, C., M. Terras, P. Hungtington, and N. Pappa. "If You
Build It Will They Come? The LAIRAH Survey of Digital
Resources in the Arts and Humanities." Paper presented at
Digital Humanities 2006, Paris Sorbonne, 5-9 July 2006. 2006.