Institute of Polish Language - Polish Academy of Sciences
From the Lexicon to the eLexicon
The first fascicle of the Lexicon mediae et infimae Latinitatis Polonorum (Dictionary of Polish Medieval Latin, henceforth LMILP) was published in 1953. The project aims at providing an exhaustive account of the Latin vocabulary used on Polish territory during the Middle Ages. Addressed to a scholarly public, the dictionary does not make many concessions to a less advanced user. Information is often conveyed only indirectly, by means of typographic devices or is left to be inferred by the reader. The project of retro-digitization of the LMILP started in the mid-2011 and was
completed by the mid-2014. The web application, although completed, is still subject to modifications and
The XML encoding of the dictionary was by no means an ultimate goal of the project. Instead, the idea was to make the rich content of the LMILP fully searchable through a user-friendly interface. This objective, however, has deeply influenced the XML schema design. The TEI (TEI Consortium 2016) was chosen as an annotation standard because at the time the work started it had been already employed in major electronic lexicography projects (Lewis-Short by the Perseus Project; DuCange by the ENC). The popularity that the standard had gained among scholars contributed to emergence of lively community which produced documentation and use cases which supplemented the “Dictionaries” chapter of the TEI Guidelines. Also, the very fact that the TEI Guidelines offered a set of ready-to-use tags for the description of lexicographic content was not without significance. Finally, the TEI XML was supported by major software providers, an important factor for the project in which adaptation of existing rather than writing new software was planned.
The paper dictionary was scanned and the output of the OCR program (Abbyy FineReader 11) was exported to ODT files; from each a content.xml file was extracted and then applied a series of XSL transformations. The main goal was to simplify styles that were automatically generated by the OCR software. Resulting XML files underwent second phase of XSL processing in which constitutive parts of the dictionary, such as entry, headword, sense definition etc., were encoded. The output XML files were again retranslated into ODT format: entries were encoded as paragraph styles, other tags were represented as character styles. In the next phase of the project the lexicographers started to proofread OCR text and correct errors of automatic annotation. This task was performed with the help of LibreOffice Writer exclusively without annotators being actually conscious of the underlying XML structure. From the practical point of view, annotation consisted in verifying whether automatic XSL processing produced correct styles; if this was not the case correct style had to be applied, as in standard text processing task.
B • 13 -
■□ass s -
■es New Roman
DISPERSIO s. DISPARSIO, -onis.£ | Th. BI. |S. I . A. N. I.:>r„;,r \-'vrum: rozrzucenie, roz\ sianie; actus disper^ndi, dissipatio StPPP VIII |p. 229 (a. 1387): pro d-e frumenti. AKapSad II p. 94 (a. 1502): interrogatus super articulo de |d-e hostiarum consecratarum, quibu parochianos communicare. Cf. Th. V1, 1412, \42 sjpp N. conc£ foeni i. cpfyenum XIII p. 410 (a. 1463): ministerialis ... |post recepcionem feni vestigiis alias «koleg^ parsionem parwam feni recepti inse|cutus est... usque in bona tua. 2. Jwminum | a. rozproszenie; eiectio, dissipatio, |V cmetijonum: StPPP VIII p. 199 (a. 1386) : |tj
Fig. 1: “XML-unaware” annotation in the LibreOffice window
This approach allowed for reducing the learning curve to a minimum so that the team members could focus on the lexicographic content. However, it also has a serious drawback: annotation in the text editor cannot produce more complex hierarchies, since paragraph and character styles allow for representing at best two levels deep nesting.
TEI for the eLexicon
A guiding principle of the subsequent TEI annotation was to combine editorial and lexical view of the dictionary content by (1) preserving its original text and (2) storing normalized data in attributes and empty XML elements. Typographic properties of the
text, on the other hand, were not generally encoded, they are easily reconstructible though.
Automatic and manual annotation consisted in three major procedures:
a. translation: custom ODT styles (corresponding to elements of dictionary structure) were “translated” into respective TEI elements or attributes;
b. grouping: deeply nested XML structure was produced from flat annotation;
c. enrichment: implicit information was made explicit.
The paper justifies some of the annotation choices. Special attention is given to the peculiarities of encoding a scholarly lexicographic work.
1. <entryFree> element was chosen as a container for dictionary entries.
2. Essential features of the dictionary macro-and microstructure are encoded as:
<form>, <orth>; <gramGrp>, <gen>,
<iType>, <pos>; <etym>, <lang>, <mentioned>; <cit>, <bibl>, <biblScope>,
<date>, <quote>; <sense>, <usg>, <def>, <gloss>; <xr>, <ref>; <lbl>; <re>, <certainty>, <oVar>, <note>.
3. Content and form peculiarities of the LMILP are reflected in respective attributes. So, for example, functional variation of the entries is represented in the @type attribute of the <entryFree> and can take one of the following values: main, xref, hom.
4. The TEI schema was only lightly customized: unused elements were deleted; a few content restrictions were overcome.
Grouping: adding depth
The flat entry structure had to undergo heavy XSL
processing, so deep nesting typical of scholarly dictionaries could eventually emerge. Relative ease of the XML-unaware manual annotation resulted in timeconsuming post-processing. The xsl:for-each-group
XSL function was employed in order to structure:
1. citation groups:
<ref type="siglum" target="fons:RachJag">RachJag </ref> cbiblScope type="pp" n=ll215">p. 215 </biblScope>
(<time when="1395">a. 1395</time>) </bibl>:
<quote>pro cmilestone unit "lb" xml:id="2.1.3"/>VIII vlnis
<«gloss xml:lang="pl-x-med">pokoczin</gloss»> grisei ad c-um dni regis cmilestone unit="lb" xml:id="2.1.4"/>sub athlas ponendum. cmilestone unit="lb" xml:id="2.1.5"/>
2. etymological groups:
(cmentioned xml:lang="la-x-cla">caput</mentioned> <certainty cert="low" locus="value"/>?)
3. PoS and grammar groups:
csense orig="2." n="2" xml :id="caballinus.2"> clabel type="numbering">2.c/label>
cusg norm="nat" type="dom" target="abbr:nat.dom">nat.c/usg> cusg type="colloc"> tricmilestone unit="lb" xml :id="2.1.38"/>
cmilestone unit="page" n="2" xml:id="2.2"/> cmilestone unit="lb" xml :id="2.2.l"/>folium c/usg>
cdef xml :lang="pl">przetacznik bobowniczekc/def>; cdef xml :lang="la"> Veronica Becca
cmilestone unit="lb" xml :id="2.2.2"/>bunga Linn.c/def> ccit type="inline">
cref type="siglum" target "tons :RFil#XXV"> RFil XXV c/ref> cbiblScope type="pp" n="282">p. 282 c/biblScope>
(ctime when="1450">a. 1450c/time>) c/bibl>
Enrichment: expanding the dictionary content
Considerable effort was put into enriching the original content of the dictionary, namely: (1) resolving references, (2) normalizing strings, (3) adding redundant and/or inferred information.
A typical reference to an exact location in the dictionary text was encoded as follows:
References to a specific entry or sense relied on the @xml:id attribute:
cref target="#caballinus.2">CABALLINUS 2</ref> </xr>
The encoding of most frequent type of references (pointing to a source of a language use example) is illustrated in the section II B 4 above.
By string normalization, we mean a set of various procedures applied in order to generate a lexical view of the dictionary content. Standardized strings are usually stored in @norm attributes of such elements as language or usage labels, prepositional and inflec-
tional patterns etc. Their primary goal is to enable unified search that would be agnostic of the exact formulation of the paper dictionary. For example, when looking up philosophy-related terms one should be able to retrieve them no matter whether they have been marked with a phil. label or with more verbose formula in textibus philosophicis “in philosophical texts”, as both are annotated as @norm= ”phil”. The second major goal of the normalization was to render chronological information consistent and machine-readable. Its proper annotation should reflect the fact that many medieval texts cannot be dated but only approximately. Apart from some obvious cases (@when attribute stores a year date, for example <time when="1450">a. 1450</time>) the LMILP employs:
1. century dates
(<time notBefore="1401" no-
tAfter="1500">saec. XV</time> )
2. imprecise dates in year (<time no-tAfter="1120">ante 1120</time>) or century format (<time notBefore="1401" no-tAfter="1450">saec. XV in.</time>).
Making information explicit
Finally, substantial effort has been devoted to making explicit what is not expressed directly in the paper dictionary, but left to be inferred by an expert user. In the LMILP, this is the case, for example, of a part of speech label which is provided for adverbs or conjunctions, but is normally omitted from verb or noun entries. Empty elements have therefore been created and their attributes filled with the inferred content. So, in a typical case, an element <pos norm="subst"/> would be appended to a <gramGrp> group whenever the paper dictionary informs about a word's part-of-speech only indirectly, by means of a gender label (f. for Lat. femininum) or inflectional ending typical of nouns (-ae):
The Dictionary Web Application
1. Various types of indexes available in the eX-ist-db allow for efficient retrieval of content from deeply nested dictionary files and dispersed textual data.
2. A templating system allows for fine-grained web presentation of the XML content.
3. A URL rewriting engine supports a logical system of dictionary content access.
4. An out-of-the-box RESTful API exposes lexicographic content to external applications.
In the conclusion, I will also point to some difficulties that I have encountered and which have mainly to do with handling application's state, a crucial feature for multi-language tools which require storing user search results.
Nowak, K. (2014). ‘The eLexicon Mediae et Infimae Latini-tatis Polonorum. The Electronic Dictionary of Polish Medieval Latin'. In The User in Focus. Proceedings of the XVI EURALEX International Congress: 15 - 19 July 2014, Bol-zano/Bozen, edited by Andrea Abel, Chiara Vettori, and
Natascia Ralli, 793-806. Bolzano: EURAC Research.
TEI Consortium. (2016). TEI P5: Guidelines for Electronic
Text Encoding and Interchange. Version: 3.0.0.
If this content appears in violation of your intellectual property rights, or you see errors or omissions, please reach out to Scott B. Weingart to discuss removing or amending the materials.
Hosted at McGill University, Université de Montréal
Aug. 8, 2017 - Aug. 11, 2017
438 works by 962 authors indexed
Conference website: https://dh2017.adho.org/
Series: ADHO (12)