SciFiQ and “Twinkle, Twinkle”: A Computational Approach to Creating “the Perfect Science Fiction Story”

1. Introduction

In Fall 2015, we were approached by author (and prominent DH-skeptic [2012]) Stephen Marche with a request that we help him use computational textual analysis to allow him to write “the perfect science fiction story.” His specific request was for a set of “rules” to follow in composing such a story. On consultation with Marche, we devised an approach that would derive “rules” related to theme (using topic modelling) and style (using a variety of techniques, including our own original work on quantifying style) from a hand-selected corpus of Marche’s 50 favourite science fiction stories. In Fall 2016, we sent Marche a list of 14 thematic rules and created a system to provide real-time stylistic feedback on his efforts to meet a set of 24 stylistic targets. In December 2017, the resulting story, “Twinkle Twinkle,” was published in the popular technology magazine
Wired
alongside a set of detailed footnotes explaining and reflecting upon the process of its creation. Whereas Marche and his editors viewed the rule-creating process as “objective” and the publishing professionals interviewed in the piece complained that the resulting story lacked “humanity,” the process in fact blended computational analysis and human interpretation. We regard it as an instance of computer-assisted “creative deformance” rather than “robotic authorship.”

2. Thematic rules

Below is a selection of the 14 rules we sent to Marche:

1.
The story should be set on a planet other than earth.

2.
The story should thus NOT be set in space itself.

3.
On this planet, there should be an existing, non-human civilization. This civilization should have a hierarchical social structure with a powerful ruler. Inhabitants of this alien civilization should be given clearly non-human names. The protagonists of the story should be humans who are directly observing this civilization from a certain distance and do not consider themselves part of it.

4.
The story should be set in a city. The protagonists should be seeing this city for the first time and should be impressed and dazzled by its scale.

5.
Part of the action should unfold at night during an intense storm.

6.
Include a pivotal scene in which a group of people escape from a building at night at high speed in a high-tech vehicle made of metal and glass that is large enough to live in for an extended period (it should have a bed in it, for instance).

9.
Include a scene set on a traditional earth farm, with apple trees and/or corn fields. In this scene, a mother and father are present. (Given the other rules, this is most likely a flashback to a protagonist’s childhood, but I leave the details to you, of course.)

10. Include extended descriptions of intense physical sensations and name the bodily organs that perceive these sensations.

13. DO NOT focus on conventional domestic family life. Marriage should not be a theme. No scenes should depict a conventional bourgeois family (especially a happy bourgeois family) at the dinner table.

These rules were derived as follows. First, we assembled a corpus of some 4,000 texts, of which approximately 3,000 were fiction
other
than science fiction, 1,000 were science fiction, and 50 were the stories hand-selected by Marche. All texts were processed in GutenTag (Hammond and Brooke 2017). Next, we performed topic modelling using Mallet in R (400 topics, 500-word chunks, nouns only). Once this was complete, we examined all the topics that distinguished Marche’s corpus from both comparison sets, positively and negatively, with statistical significance values of p < 0.05. Some of these topics were easily converted into thematic rules, such as topic 199 (Fig. 1), which became the basis of rule 5. Less legible topics required a more creative, less “objective” approach. This was the case for topic 33 (Fig. 2), the topic which most clearly distinguished Marche’s hand-selected corpus from both comparison groups, and which became the basis of rule 6. To preserve the aura of computational objectivity that he craved, Marche was not shown the topic modelling word clouds, and the process by which the rules were devised was not explained to him in detail.

Figure 1:
Topic 199, the basis of rule 5.

Figure 2:
Topic 33, the basis of rule 6.

3. Stylistic rules

Marche began composition once he had received the list of thematic rules. We instructed him to insert his drafts into an online system we had devised, SciFiQ, which would provide stylistic feedback based on 24 criteria:

Literariness
Abstractness
Objectivity
Colloquialness
Concreteness
Subjectivity
Positivity

Text length

Average word length
Average sentence length
Average paragraph length
Average variance in sentence length
Average variance in paragraph length
Average commas per sentence
Percentage of Latinate words
Nouns per 100 words
Verbs per 100 words
Adjectives per 100 words
Adverbs per 100 words

Lexical density

Speaking characters
Percentage of text which is dialogue

Percentage of dialogue by female characters

Major named locations

All criteria were calculated using GutenTag, which was configured to identify parts of speech, structure, and named locations (using LitNER [Brooke et al. 2016b]); to distinguish narration from dialogue; to identify individual characters and their gender; and to tag each word for stylistic properties (Brooke et al. 2016a) and sentiment polarity. SciFiQ displayed results for each of these criteria, colour-coded to indicate how close Marche was to his targets. Only once the values were within 0.5 standard deviations of the mean value for all stories in the 50-story corpus would the value read green; otherwise they would read purple (too low) or red (too high). Once Marche had all values in the green range (with the exception of story length, which was at the discretion of his editor), composition was considered complete. The story was not modified at all during the editing process for
Wired.

Fig 3:
SciFiQ in composition mode.

Fig 4:
SciFiQ’s analysis report.

4. Discussion

We approached this project with several aims in mind. First, and most prosaically, we regarded it as a means of validating our work on quantifying style (Brooke et al. 2016a). If our system told Marche to make his style less literary, for example, and he made a series of edits based on his intuitive concept of literariness, would our system respond in a way that he regarded as intuitive? In practice, Marche found that our system’s analyses did correspond to his intuitions. We thus consider our approach to style to be further validated. Second, we viewed the project as a way of testing the extent to which a set of tools designed for computational textual
analysis
(some developed by ourselves) could be useful in
composition
. Following critics like Jerome McGann (2001) and Stephen Ramsay (2011) we were interested in computational tools’ ability to “deform” the literary work in such a way as to model a helpful and non-traditional form of composition. In this, too, we believe the project was successful. In the notes he supplied in
Wired
, Marche reported, “the algorithm affected the story much more than I would have thought,” noting in particular the manner in which conflicting rules (1 and 9) presented welcome imaginative challenges, and in which the SciFiQ system demanded a “Rubik’s cube”-like writing process whereby alterations in one part of the story required counterbalancing edits elsewhere.

Of course, we understood that the project would be received in a manner not entirely in keeping with our intentions. Whereas we regarded the project as a somewhat playful creative disruption of the conventional process of composition, Marche presented SciFiQ as “a computational system that [would] optimize [his] prose” and “make [him] better at his job.” In a sidebar,
Wired
asked two prominent literary editors (Andy Ward, editor-in-chief of Random House, and Deborah Treisman, fiction editor of
The New Yorker
) to provide general feedback on Marche’s story, “without knowing who (or, more specifically, what) wrote it.” The implication that the story was
written by
an algorithm is inaccurate (indeed,
Wired
credited it to Stephen Marche, not to SciFiQ). Further, the claim that opinions were solicited “blind” appears disingenuous given that Ward’s primary complaint was that the story “doesn’t sound human” — an unusual comment unless one has been told that computers were somehow involved in the story’s composition. From the perspective of
Wired
, then, the story’s appeal as well as its danger lay in the notion of computers taking over the essentially human act of storytelling.

5. Conclusion

The “Twinkle, Twinkle” project will likely be received by the public as another instance in the longstanding debate about the proper role of machines in storytelling. Yet for us, its significance is rather different. It served as an opportunity to use tools and approaches designed for the analysis of literature in creative production; to validate our 6-dimensional approach to style; and to reach across the divide that continues to separate academic work on literary analysis both from contemporary creative writers and their popular audiences.

Bibliography

Brooke, J., Hammond, A., and Hirst, G. (2016a). Using Models of Lexical Style to Quantify Free Indirect Discourse in Modernist Fiction.
Digital Scholarship in the Humanities,
2(2): 1–17.

Brooke, J., Hammond, A., and Baldwin, T. (2016b). Bootstrapped Text-level Named Entity Recognition for Literature. In
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL ’16).

Hammond, A., Brooke, J. (2017). GutenTag: A User-Friendly, Open-Access, Open-Source System for Reproducible Large-Scale Computational Literary Analysis. In
Digital Humanities 2017 Abstracts. Montreal: DH2017, pp. 246-249.

Hammond, A., Brooke, J. (2016). Project Dialogism: Toward a Computational History of Vocal Diversity in English-Language Literature. In
Digital Humanities 2016: Conference Abstracts. Jagiellonian University & Pedagogical University, Kraków, pp. 543-544.

Marche, S. (2017). Twinkle, Twinkle.
Wired, (Dec 2017): 108–115.

Marche, S. (2012). Literature Is Not Data: Against Digital Humanities.
Los Angeles Review of Books.

McGann, J. (2001) Radiant Textuality: Literature after the World Wide Web
. New York: Palgrave.

Ramsay, S. (2011). Reading Machines: Toward an Algorithmic Criticism. Urbana, IL: University of Illinois Press.