This is the kind of thing I just don’t understand the value or use of:
This paper is the report of a study conducted by five people – four at Stanford, and one at the University of Wisconsin — which tried to establish whether computer-generated algorithms could “recognize” literary genres. You take David Copperfield, run it through a program without any human input – “unsupervised”, as the expression goes – and … can the program figure out whether it’s a gothic novel or a Bildungsroman? The answer is, fundamentally, Yes: but a Yes with so many complications that make it necessary to look at the entire process of our study. These are new methods we are using, and with new methods the process is almost as important as the results.
So human beings, over a period of centuries, read many, many books and come up with heuristic schemes to classify them — identify various genres, that is to say, “kinds,” kinship groups. Then those human beings specify the features they see as necessary to the various kinds, write complex programs containing instructions for discerning those features, and run those programs on computers . . . to see how well (or badly) computers can replicate what human beings have already done?
I don’t get it. Shouldn’t we be striving to get computers to do things that human beings can’t do, or can’t do as well? The primary value I see in this project is that it could be a conceptually clarifying thing to be forced to specify the features we see as intrinsic to genres. But in that case the existence of programmable computers becomes just a prompt, and one accidental, not essential, to the enterprise of thinking more clearly and precisely.
The primary value I see in this project is that it could be a conceptually clarifying thing to be forced to specify the features we see as intrinsic to genres.
I sincerely doubt it can help with that, Alan. So far as I can see what the programmers did was feed the computer a list of works and then have it select others based on perceived similarity. Nowhere did they give the computer, or it them, a list of explicit criteria that were being used for the sorting.
Prof. Jacobs,
You ask, "Shouldn't we be striving to get computers to do things that human beings can’t do, or can't do as well?," but from the material contained within the study's PDF (and not just quoted in the abstract), this is precisely what the researchers are doing. Having isolated the features that we see as intrinsic to genre, the researchers apply a number of different methods to see how those genres track. These results shouldn't necessarily be surprising, for after all, as one of the pamphlet's authors points out in a separate essay in MLQ, "The Slaughterhouse of Literature," if we control for those features acknowledged to belong to a particular genre, the results can only ever reflect the choices we've already made. The value of the study, however, is to look at the moment where a variety of different analytic methods "agree" yet in opposition to native human intuition. Section 4 notes one computerized attempt to isolate the "most Gothic" page within the specified corpus, returning – surprise, surprise – a page from Ann Radcliffe. Yet the features that the computer seizes on are not the same gothic features that are obvious to a human reader. What this reveals, to quote the authors in their own words, are the unseen "consequences of higher-order choices…the /effects/ of the chosen narrative structure" (8). Overlaying these methods allows us to "'see' the space of the gothic," or the link between narrative choices and lexical features with greater clarity (ibid).
The "digital humanities" constantly attracts attempts to prove specific theses – the authorship of an anonymous Renaissance play, or the fact that Scott and Austen are more "alike" than they are to Dickens – yet with no real standardization of methods. Every study involving computerized data (at least every one that I've ever seen) requires pages upon pages explaining the statistical validity of an approach used just in that one study. At its broadest, the "experiment" in "quantitative formalism" offered here is an attempt to isolate best practices for any other types of analysis going forward. By most definitions, that would not only constitute a contribution to knowledge, but a valuable.
Brandon, as far as I can tell your last two sentences largely echo what I say in my last two sentences. What I call "gaining conceptual clarity" you call "isolating best practices." Or am I misunderstanding you?
It's possible that I'm misunderstanding you. When you wrote "gaining conceptual clarity," I assumed that you meant that the experiment was valuable primarily as a thought exercise: that is, by inputting a bunch of variables into a computer, we're compelled to reflect on what those variables ought to be. The data entry – rather than the data analysis – allows us, as humans, to think more deeply about genre. If this is the case, then the data analysis component is already irrelevant.
My case, however (and that, I believe, of the authors in question), is that this kind of analysis can expose us to new appreciations of genre (or whatever else) that humans alone can't intuitively acknowledge. This is what we do in other walks of analysis: isolate certain features shared by a group, and then look at that group more closely to see what traits they have in common. Thus we can say that while democratic republics are defined primarily by their form of government, most also share the additional trappings of civil society, or perhaps even something arbitrary and unpredictable, like access to running water. This sort of generalization (which may be correct for all I know) is possible simply by examining the surface features of the societies in question, or else, with the aid of a computer, going over raw statistics. But the problem, when analyzing texts, is that looking beyond the surface is hard. In the same way that we generally let computers process demographic information for social science analysis, human attention can only be counted upon to notice so much in the ordinary processes of reading. What the researchers are maintaining is that there is a way to apply an analytic approach to texts, and then providing a sampling of case studies for how to get there.
When I say "isolating best practices," I don't mean simply "determining the best ways to think about genre," but rather determining which types of analysis are most likely to isolate areas of interest beyond the obvious. Given the sheer amount of data in play, it is of course possible to restrict analysis only to return the desired results. Thus this attribution of "The Reign of Edward III" to Shakespeare almost arbitrarily determines that the mark of authorship is found in concatenations of three word phrases. But by applying a host of different methods, the researchers here seem after the broad types of analysis that allow a computer to tell us something interesting about a text that we didn't tell it in advance. If we're to isolate "what we talk about when we talk about genre," looking beyond the surface needs to be an intrinsic rather than instrumental part of that process. "Gaining conceptual clarity," here is something that does then require the computer's intervention.
Brandon, your second comment got caught in Blogger's spam filter. Sorry about that. In the meantime I wrote a second post, which I probably would have written differently if I had seen your comment first. While your explanation is, I think, clearer than the original pamphlet, I still feel — and perhaps this is just my own ignorance — that there's a certain vagueness to the whole enterprise, at least some of which stems from the use of familiar literary-critical terms to describe new methods of analysis that those terms don't really fit.
For instance, you speak here of "looking beyond the surface," and the pamphlet's authors speak also of metaphors of "depth," but I don't know what that means. It seems to me that none of these approaches have anything to do with surfaces and depths, but rather just computer programs that do pattern recognition differently than we do — or rather, differently than we do consciously, since the program can only use the algorithms that we write for it and therefore is wholly dependent on what we already perceive, even if it processes those perceptual categories somewhat differently.
All that to say that I'm still finding the whole enterprise hazy and undefined, for reasons I explain in the other post.
[brett and melanie] is Google Instant safe.
[brett and melanie ] is not Google Instant safe.
So now were going to have *computers* define genres? Why would a Uni Lit department want to play at being Google?
Shouldn't we be striving to get computers to do things that human beings can’t do, or can't do as well?
Everything we ask computers to do is something human beings can do. Computers can just do it faster.
If this process works the way we use computers for other things, a possible use would be that if we develop a process that lets computers "read" and roughly sort books the way a human would sort them, we could turn that process loose on an enormous pile of unsorted texts and get some (preliminary) classifications much faster and cheaper than if we hired a thousand English graduate students to read and sort them all.
Another possible use would be that computers might discover relationships that humans miss because we're distracted by other details or preconceptions. Maybe one of Jane Austen's novels scores surprisingly high in the "Western" genre.
That's all the theory behind this, I guess. I'm very skeptical that we're anywhere close to developing computer programs that can usefully sort books into genres, and I'm not sure that it would be very worthwhile to find out that a computer finds that Pride and Prejudice is more like a Western than any of Austen's other novels or something like that.