Making Waves: Algorithmic Criticism Revisited

In Reading Machines: Toward an Algorithmic Criticism, Stephen Ramsay suggests that computational literary studies remain marginalized because they lack “bold statements, strong readings, and broad generalizations” (2011: 2). They are too cautious, too scientific, to interest literary critics, who value opening texts to new interpretations over solving problems (10-11). Ramsay suggests that a feminist discussion of The Waves challenges algorithmic criticism: “literary critical arguments of this sort do not stand in the same relationship to facts, claims, and evidence as the more empirical forms of inquiry. There is no experiment that can verify the idea that Woolf’s . . . ‘elision of corporeal materiality’ exceeds the dominant Western subject” (7). Although many critical claims are computationally intractable, Woolf’s “elision of corporeal materiality” surely has textual implications that might be tested computationally. Literary criticism’s problematic relationship to facts, claims, and evidence seems more like a bug than a feature, but here I want to re-examine and interrogate Ramsay’s algorithmic provocation.

Three male and three female characters in The Waves speak in alternating monologues, an experimental technique that has invited critical comment about what axes of difference or unity characterize the novel. “The ‘problem’ . . . with Woolf’s novel is that despite evidence of a unified style, one suspects that we can read and interpret it using a set of underlying distinctions. We can uncover those distinctions by reading carefully. We can also uncover them using a computer” (Ramsay 2011: 10-11).

Ramsay treats the six monologues as a corpus of documents and investigates them with tf-idf, from the field of information retrieval: tf*(N/df). This, he suggests, should identify each monologue’s characteristic words more effectively than a traditional word-frequency list. Tf-idf is the term’s frequency (tf) multiplied by the total number of documents (N; here 6) divided by the number of documents containing the term (df; document frequency). Tf-idf reduces the importance of function words and increases the importance of speakers’ characteristic words because the frequencies of words used by only one speaker are multiplied by six (6/1), while the frequencies of words used by all six speakers are multiplied by one (6/6) (Ramsay 2011: 11). After identifying each speaker’s most characteristic words, he reveals that he has actually used the formula, 1 + tf *log(N/df), which includes a log function (reducing the effect of a word’s appearance in only one speaker), and adds 1 (preventing the measure from becoming negative). The purpose of the alterations “is not to bring the results into closer conformity with ‘reality,’ but merely to render the weighting numbers more sensible to the analyst” (Ramsay 2011: 15). Yet the variants are not “merely” at the whim of the analyst; they have testable consequences.

But let us travel a bit further with Ramsay. He presents the words with the highest tf-idf scores in Louis’s monologue (listed in Fig. 1, along with his tf-idf scores, my tf-idf scores, and their frequencies), and suggests that “Few readers of The Waves would fail to see some emergence of pattern in this list” (12). For example, western seems to echo Louis’s concern about his Australian accent, and England (all top 25). But actually western, wilt, and thou appear only in Louis’s quotations from a sixteenth-century poem. Ramsay’s provocation ignores some interesting questions: should Louis’s quotations be considered his speech (and retained?), or the anonymous author’s (and omitted?).

Fig. 1: Louis’s Most Characteristic Words

Trying to recreate Ramsay’s analysis reveals further interesting points. The mismatched tf-idf scores (bold) reflect different word frequencies. His score for beast requires 6 occurrences, not 5, but the 6th is in the omniscient narration). His score for accent requires 13 occurrences, not 14, but Rhoda’s most characteristic words include them– and accent occurs once as accent–, which presumably reduces his count by 1. (What constitutes a word is a surprisingly complex question, but treating accent– as a word seems odd.) The rarity of the words shows how strongly tf-idf privileges words limited to one character (only accent appears in 2). Ramsay’s intervention raises interesting questions: What does it mean to choose this algorithm? How do the results affect our emerging reading of The Waves? (Ramsay 15). But how to answer these questions?

Even the identification of characteristic words is problematic. Tick and hoot both occur 4 times, only in Bernard, but all 8 occurrences are in 3 consecutive sentences. How “characteristic” is that? Low occurs just 5 times, “only” in Bernard, yet it also occurs in the omniscient narration. Analyzing only the 6 characters seems reasonable, but should Bernard’s characteristic words also occur in the narration? (Some consider Bernard to be modeled on Woolf herself [Ramsay 2011: 13].) Including the narration seems both intriguing and problematic, not least because it is not dialogue. Doing so removes low, canopy, bowled, and brushed from Bernard’s most characteristic words and beast, steel, and discord from Louis’s. What questions does this raise?

Most algorithms for computational approaches come from authorship attribution, where ostensibly correct answers exist. But Ramsay is right that the existence of “correct” answers to questions like “Do the men and women speak differently?” or “Do the characters have distinct and consistent voices?” is precisely at issue. Examining The Waves in the light of Ramsay’s provocation raises so many intriguing questions that they cannot all be addressed here. But we can approach the question of character individualization by using a radical deformation. I randomly sorted the lines of the six monologues, then selected the first 6067 words of each, the length of the shortest monologue (Susan’s). I identified the 50 most characteristic words using Ramsay’s tf-idf formula, and tested how well they group with the remainders of the longer monologues using cluster-analysis, starting with all 300 words (in descending tf-idf order), then reducing the number gradually. The best result, for the 20 most characteristic words, is shown in Fig. 2.

Fig. 2: Tf-idf and Character-Individualization

Bernard’s and Louis’s sections group together, while Neville’s and Rhoda’s fail (Jinny and Susan have too little text for 2 sections). A simple word frequency list, however, correctly groups all 4 in many analyses (see Fig. 3, based on the 300 mfw), providing a tentative answer to the question of distinct voices.

Fig. 3: The Most Frequent Words and Character-Individualization

Selecting the 50 most characteristic words for each monologue using Zeta (Craig and Kinney 2011), also produces many perfect results (see Fig. 4, based on the 200 most characteristic words). This very different method, which measures consistency of use rather than frequency, confirms the distinctness of the voices. Finally, testing the six characters in 2,000-word sections with 2-grams (based on the six full monologues) also yields many completely correct clusters (see Fig. 5 for an analysis based on the 900 mf2Grams).

Fig. 4: Zeta and Character-Individualization

Fig. 5: 2Grams and Character-Individualization

Ramsay suggests that treating the question of whether the six characters in The Waves share “the same stylistic voice” as a problem to solve is a “category error,” and that the proper question–one computers cannot answer–is “Can I interpret (or read) it this way?” (2011: 9-10). Critics still can read the novel as a single stylistic voice, and the six monologues undoubtedly share many characteristics, but, in spite of a host of very interesting remaining questions about the status of algorithms, arguments, and evidence, it seems reasonable to make the bold claim that there are six distinct character voices in The Waves.

Ramsay’s provocative intervention is valuable for forcing us to re-examine our methods and focus on questions of interest to traditional literary scholars. But further analysis of his provocation and his algorithm suggests that more attention to the text, to the nature and function of the algorithms, and to method may prompt bold claims that rest on a sounder foundation. Further work will help us explore the boundary between computationally-tractable and computationally-intractable questions and the significance of that boundary.

References
Ramsay does not analyze, then interpret, a method criticized by Fish (2012). He begins with a literary judgment, then investigates it. Despite Fish’s criticism, both methods seem valuable.

Ramsay actually uses (1+LN(tf))*LN(N/df), not 1+tf *log(N/df). Many tf-idf formulas exist; for the 4 I tested, 20 of the 25 most characteristic words are the same.

It also reveals that Ramsay has (not unreasonably) omitted Bernard’s final long “summing up” chapter, as I have also done.

The characters do not group completely by gender in the graphs above, also suggesting a tentative answer to “Do the men and women speak differently?” that is different from Ramsay’s.

Craig, H., and A. Kinney. (2010). Shakespeare, Computers, and the Mystery of Authorship. Cambridge UP.

Hoover, D. L. (2007). “The End of the Irrelevant Text: Electronic Texts, Linguistics, and Literary Theory,” Digital Humanities Quarterly 1(2).

Ramsay, S. (2011). Reading Machines: Toward an Algorithmic Criticism. Urbana: University of Illinois Press.

Fish, S. (2012) “Mind Your P’s and B’s: The Digital Humanities and Interpretation,” The New York Times, online, January 23..