ChatGPT is terrifyingly good at lots of things, but not at understanding literature—and we have the test scores to prove it. It’s been widely reported that the latest version of the artificial intelligence chatbot was assigned a number of standardized tests, including the SAT, the Bar exam, and a variety of Advanced Placement tests, and excelled in all, except for the two Literature APs, where it received sub-middling grades.
We literature lovers crowed, of course, but having suffered the scorn of more “practical” people all throughout our lives, we should know better than to be complacent. As the editor of the Bloom’s Literature database here at Infobase, I’m well aware that educators are quite reasonably afraid that students soon—or right now—will start plagiarizing their English essays from ChatGPT. So I decided to test ChatGPT’s literature skills by asking the most widely available version about Shakespeare. Its successes and failures both impressed and reassured me. ChatGPT has its virtues, but it’s unable to reproduce the one thing that makes text worth reading.
For those of you still more puzzled than disturbed by ChatGPT, here’s an explanation. ChatGPT is an artificial intelligence chatbot that has been provided with a store of knowledge encompassing vast swaths of the internet and trained by digital experts in the art of conversation. You can ask it questions, watch as it provides answers within seconds, and then respond by asking it to dig deeper in its analysis, or change its perspective, or emphasize different aspects of its argument. Some of the immature among us, including students, journalists, and Twitter personalities, like to encourage ChatGPT to sound silly or psychopathic. Others have found therapeutic value in an exchange with ChatGPT. But most users don’t care about its supposedly realistic conversational qualities. What they know is, when they ask ChatGPT to write something, it follows directions. And the resulting text sounds confident and human. And most importantly (and alarmingly) of all, the text can be copied and pasted anywhere—including Google Classroom.
ChatGPT and Othello
I began my Shakespearean colloquy with ChatGPT by asking it about a line from Othello. Early in the play, Othello comes across some angry, armed Venetians, itching to fight. He silences them with this: “Keep up your bright swords, for the dew will rust them.” Literary critics have praised the line as a striking example of Othello’s power of command, but when I first read it, I found it a little puzzling. My copy of the text notes that “keep up” means “put away” or “keep out of sight,” which is helpful but not exactly comprehensive. Here was my exchange with ChatGPT:
Me: What does Othello mean by “keep up your bright swords, for the dew will rust them?”
ChatGPT: In Shakespeare’s play Othello, the character Othello says the line “keep up your bright swords, for the dew will rust them” in Act 5, Scene 2, when he enters the scene where the final conflict between the characters is about to take place.
The line can be interpreted in a couple of ways, but essentially, Othello is urging the characters to stop fighting and put away their weapons. The phrase “bright swords” refers to the weapons themselves, which are likely shining in the moonlight, and the word “rust” implies that leaving them out in the damp night air will cause them to deteriorate over time.
On one level, Othello may be suggesting that the conflict has gone on long enough and that continuing to fight will only lead to further destruction and harm. On another level, he may be acknowledging the futility of the fighting, implying that it is pointless to continue when the outcome is already certain. Ultimately, Othello’s line serves as a warning to the other characters that violence and aggression will only lead to their own downfall.
In one way, this response is impressively human-sounding and more or less correct, at least regarding the line’s general effect on readers. In another, it’s absurd. First of all, ChatGPT’s initial paragraph is totally wrong. Othello says the line early in the play, in Act I, Scene 2, not in Act 5, Scene 2, as ChatGPT claims, and certainly not during the play’s “final conflict.” The play has four more acts to go. Any student who wrote that paragraph would be immediately exposed as a faker.
ChatGPT’s outright mistakes—which designers have dubbed “hallucinations”—seem to result from the misapplication of common patterns that its algorithm has detected within its vast resources of online text. Here I can only speculate that it has been noticed that people prefer to analyze “final confrontations” rather than Act 1 encounters. Its makers insist that ChatGPT can learn from these mistakes, and indeed users report that the more advanced versions, already in development, have eliminated most of them.
But the problem goes beyond hallucinations. Even where ChatGPT is correct, it’s boring. Its claim that “Othello may be suggesting that the conflict has gone on long enough and that continuing to fight will only lead to further destruction and harm” is true but obvious and tediously expressed.
By contrast, here is the literary critic Harold Bloom, in an essay available in the Bloom’s Literature database, explaining how the line in question embodies Othello’s eternal appeal:
When we think of him in his glory we remember his ending a street battle with one line of marvelous authority:
Keep up your bright swords, for the dew will rust them.
“Sheathe or die” would be the reductive reading…
What ChatGPT explains in three forgettable paragraphs, Bloom explains in three memorable words: the line means “Sheathe or die.” ChatGPT’s responses, based on its masses of information and guided by its purpose, which is to please its interlocutors, tend to be ordinary in content and formulation. A memorable, original phrase like “Sheathe or die” would never occur to it.
ChatGPT and Hamlet
I next asked ChatGPT to explain a more famous speech, Hamlet’s “To be or not to be” soliloquy.
Perhaps because this speech has been more often analyzed, the bot was able to avoid obvious whoppers. After quoting from the speech and explaining some specific phrases, it declared:
Ultimately, Hamlet concludes that the fear of the unknown is what keeps people from taking their own lives, and that it is this fear that makes life worth living. He decides that he will not take his own life, but will instead continue to suffer the “slings and arrows of outrageous fortune” in the hope that someday, things will get better.
Overall, the “to be or not to be” speech is a reflection on the nature of existence, the human condition, and the struggle to find meaning and purpose in a world that can often seem cruel and senseless.
The second paragraph is a true and useful, if commonplace, explanation of the speech. The first paragraph, however, is utterly ridiculous. Though it’s true enough to say that “Hamlet concludes that the fear of the unknown is what keeps people from taking their own lines,” the bot’s succeeding pronouncement, “it is this fear that makes life worth living” makes absolutely no sense. Fear of the unknown makes life worth living? Is that why we get up in the morning? What about say, love, joy, charity, family, a sense of purpose, etc.? No human would ever make such a claim, except perhaps someone completely addled by the study of endless reams of online self-help discourses—which ChatGPT has been forced to do.
And here in a nutshell is the problem with ChatGPT. Its whole purpose is to come up with a perfectly average, acceptable, reasonable response, based on what everything else in its dataset already says. When it sticks to the banal, it’s fine. When it tries to say anything original, it’s absurd.
ChatGPT and Your Students’ English Papers
The whole point of writing—any writing—is to say something new. That’s not to say that every writer needs to be James Joyce or Shakespeare. But you’re reading this piece in the hopes that it will provide some new information on the current state of artificial intelligence. You click on an email expecting it to tell you something new about a project at work, or your child’s progress at school, or the romantic life of your friend. You read a novel or a story or a poem hoping to find some plot twist, some turn of phrase, some human insight that you’ve never encountered before. Even a literary-critical essay—such as those we constantly select and add to the Bloom’s Literature database–should tell you something that you never knew before about a literary work, or never would have expressed so effectively.
ChatGPT can never replicate what we value most in writing—originality. What it does threaten is assessment of writing. Students may have original insights, but most of them are still gathering the basic skills they need to convey them, along with the tools they need to understand what they’re tasked to analyze. Being unfamiliar with Othello—its unique and tricky vocabulary, the rhythm of its lines, its strange setting and obscure allusions—students may wind up writing something, in all innocence, as ridiculous as ChatGPT’s craziest hallucinations, and perhaps in clumsier language. In such a situation, who wouldn’t be tempted to plagiarize from ChatGPT and provide something boring but true, and reasonably readable?
We parents and educators know resisting this temptation is vital for students to develop their unique talents. The Othello assignment may or not be important in itself (though I for one think appreciating Othello is important), but someday these students will have to write a memo to their manager, or an email to a local authority, or a report on a laboratory experiment or marketing study, or something on a subject that no artificial intelligence chatbot could know anything about because the issue is narrowly professional, or personal, or new, or otherwise unique.
Experts on ChatGPT and education are increasingly emphasizing the importance of ChatGPT-proof writing prompts and assessment techniques. For example, educators might ask students to connect one of Shakespeare’s scenes to events in their own lives, or to watch a video of a scene from Shakespeare and describe how the performance affects their understanding of the play. (Bloom’s Literature, with its countless videos of classic stage and film productions of Shakespeare’s plays, is perfect for such an assignment.) There are many other options. (For more advice on writing assessment in the ChatGPT era, see “The Rise of ChatGPT Can Make Student Writing Better” by Amy Dayton and Amber Buck.) The educator Anna Mills has been at the forefront of the issue and maintains an online resource list entitled “AI Text Generators and Teaching Writing: Starting Points for Inquiry.”
The most important teaching practice is to engage students and convince them of the importance of their own unique perspective. That’s easier said than done, of course, but it’s vitally important, and just one more thing a chatbot like ChatGPT can’t do.
What Is Bloom’s Literature?
The award-winning Bloom’s Literature database is rich with relevant content on the most studied and researched authors and works. Students will find exactly what they need without having to wade through an uncurated search. Educators will appreciate the thoughtful organization and important curriculum tools to help with lesson plans, assignments, and independent study.
Want to encourage your students to write their own papers instead of relying on an AI? Guide them to Bloom’s Literature‘s “How to Write about Literature,” which features specific, student-friendly guidance on how to write good essays on the most assigned authors and works,. Students will find strategies and ideas they can use to add an extra dimension to their writing.
See also:
- FREE webinar: Slam into Poetry Month with Contest Ideas from Infobase
- Celebrate 30 Days of Poetry with Bloom’s Literature
- Read Up on Banned Books with Bloom’s Literature
- FREE webinar: Literature in an Online World: Using Digital Resources to Apply Literary Skills in the Classroom
- Using Bloom’s Literature in the Classroom: Theme & Summary Guide: PDF
