When Frank Marshall White of the New York Journal contacted Mark Twain in 1897 about rumors of his failing health, the author supposedly wrote back, "The report of my death has been greatly exaggerated." Similarly, to equate the rise of artificial intelligence programs like ChatGPT with "The End of High-School English"—which is what Daniel Herman claims in a recent article in The Atlantic—is to misunderstand the purpose of literature and the complexity of literary analysis.
One of the guiding principles for budding English scholars is the famous Roland Barthes' quote—which is the subject of the 2004 AP Literary Argument prompt—that states, "Literature is the question minus the answer." When examining any complex literary work, which is by definition open to interpretation, we should welcome any view that can help us better make sense of a text—whether those opinions come from classmates, teachers, literary scholars, SparkNotes, or ChatGPT. Ultimately, any conclusion we reach will be provisional and subject to change—depending on new information, contrasting views, or additional thoughts that we may have on the text and our argument.
The fear that English teachers have of AI-generated programs is that students will no longer think for themselves and simply use something like ChatGPT to do the thinking for them. Fortunately, the quality of output from ChatGPT—at least as it is presently constructed, which I don't think is likely to change—is not capable of producing a quality literary analysis that would be acceptable in any English course that aims to prepare students for college-level writing or for success on assessments like the AP Literature Exam.
ChatGPT's failure is largely the result of what it is attempting to do: namely, synthesize all material that has been inputted into its dataset from the internet to create a consistent, coherent argument about something that is inherently subjective and open to various interpretations. Crafting a consistent, coherent argument about a complex literary work requires creative problem-solving to reconcile the inherent tensions and inevitable ambiguities within any text. The random amalgamation of divergent ideas that ChatGPT draws upon inevitably leads to inconsistency and, ultimately, incoherence.
For instance, when our Level I students read Ray Bradbury Fahrenheit 451, they first analyze Matthew Arnold's poem "Dover Beach" since Bradbury's protagonist, Guy Montag, recites the last two stanzas of the poem in the second section of the novel. To prepare students for what they will one day encounter on the AP Literature Exam, we format Arnold's poem as if it were Question 1 in the free-response section of the AP Exam:
When our students prepare to write any AP essay—whether it is an AP Poetry Analysis, an AP Passage Analysis, or an AP Literary Argument—we advise them to use Hegel's Dialectic to organize their thoughts. The first goal when analyzing a complex text is to identify the implied tension within the work (i.e. thesis vs. antithesis), which will then become the focus of the students' first two body paragraphs. Then, students will try to resolve that implied tension in their third body paragraph (i.e. synthesis) in a manner that reveals the writer's overall theme and, to use the AP's phraseology, "the meaning of the work as a whole."
When trying to identify the implied tension, students should first analyze the prompt since it often gives clues as to where the complexity might be located. For instance, in the prompt we created for "Dover Beach," students are asked to "analyze how the speaker’s observations of the external world reflect the complexity of his internal thoughts and feelings." Immediately students should identify that the observations of the external world somehow connect to the speaker's internal thoughts and feelings. Furthermore, students should recognize that those internal thoughts and feelings are "complex," meaning they are most likely mixed or ambiguous.
To help students delve into the specifics of the poem to identify the implied tension, we also provide a study guide with open-ended questions that focus on the four pillars of Style Analysis—diction, imagery, language, and syntax. Students will examine key quotations from specific lines of the poem that will require a subjective analysis that is open to various interpretations:
Let's examine the first question from the study guide:
What is the impact of contrasting the “calm” (1) sea and “tranquil bay” (5) beyond the shore to the “grating roar / Of pebbles which the waves draw back, and fling, / At their return” (9-11) on the beach?
By asking students to explore the contrasting imagery in the opening stanza, we are guiding them towards identifying the implied tension that the imagery suggests. When students try to answer this question, they might consider that the world is a complex place and that we always have the option to focus on either the beauty of our surroundings—as represented by the "calm" sea and "tranquil bay"—or the tumult and chaos—as represented by the "grating roar" of the waves that "fling" pebbles violently onto the shore. But is this the only interpretation? If we ask this specific question to ChatGPT, we receive the following output:
The contrast between the calm sea and tranquil bay beyond the shore and the grating roar of pebbles on the beach creates a sense of duality and inner conflict. The peaceful and serene imagery of the sea and bay represent the idea of beauty, calmness, and tranquility, while the grating roar of the pebbles on the beach represents the harshness and turmoil of reality. This contrast highlights the theme of the poem, which is the loss of faith and the decline of civilization, and adds to the melancholic and contemplative tone of the poem.
Let's first state the obvious about this computer-generated response: It's pretty amazing. However, when we look more closely at the analysis, it is filled with holes and inconsistencies. For instance, the idea that the "inner conflict" of the speaker—which is represented by "beauty, calmness, and tranquility" standing in stark contrast to the "harshness and turmoil of reality"—is an interesting thought, but what constitutes "reality" in this interpretation? Is our "reality" always going to be harsh and filled with turmoil? Isn't the beauty, calmness, and tranquility of the distant bay also part of the speaker's reality? How do we resolve the tension that is expressed in this "duality" of experience?
Furthermore, why does the "harshness and turmoil of reality" necessarily result in a "loss of faith and the decline of civilization"? These are bold, presumptuous claims made by ChatGPT that are not necessarily supported by the text or its own analysis. If students can connect the these claims with the text of the poem, they might have the beginnings of a compelling argument. ChatGPT does not supply those connections, however, nor does it provide evidence from the text to support its claims.
Students could theoretically cut-and-paste ChatGPT's response to this question and pass it off as their own on the study guide, but it will not help them create a consistent, coherent argument when it comes time to write the essay. Students can certainly use the information that ChatGPT provides as a starting point, but they will still need to evaluate those ideas critically and then choose the ones that make the most sense in the context of the larger argument they are trying to make about the poem and Arnold's implied intent.
When students are confused by the computer's response, they can ask ChatGPT follow-up questions. For instance, after receiving the response above, students may ask how Arnold's poem attempts to resolve the "inner conflict" created by the "duality" of existence in a way that potentially reveals the poem's overall theme. ChatGPT's response to that query is the following:
The duality between the beauty of the tranquil bay and the harshness of the crashing waves represents the contrast between appearance and reality. The poem explores the theme of the disillusionment with the world and its loss of faith and spirituality. The speaker laments the loss of faith, symbolized by the receding of the sea, and the harsh reality that has replaced it. The resolution to this duality lies in the idea that although the world may appear calm and beautiful, there is a deeper, underlying turmoil and harshness. The speaker finds solace in the idea that love, despite its impermanence, can provide a temporary escape from this harsh reality.
In this response ChatGPT claims that the "duality" in the poem is between "appearance and reality," but how is the "appearance" of something different from its "reality," and how is this interpretation supported with evidence from the text? Doesn't our perception of reality depend on what we are looking at? The "appearance" of the tranquil bay suggests that the "reality" of the world is one that is "beautiful and peaceful"; however the "appearance" of the crashing waves below suggests that reality also contains an "underlying turmoil and harshness." If both of these "realities" are true simultaneously, then how do we resolve our "inner conflict" and why would it result in "the theme of disillusionment with the world and its loss of faith and spirituality"?
Further confusion is created when ChatGPT claims that the tension of this duality is resolved when the speaker "finds solace in the idea that love, despite its impermanence, can provide a temporary escape from this harsh reality." While it is true that the speaker turns to his beloved for "solace" in the final stanza of the poem, is there any evidence that Arnold implies that their love is "impermanent" and only offers "a temporary escape" from the "harsh reality" of their lives? Perhaps there is an implication that we can alter our perception of "reality" by choosing to focus on beauty and love rather than chaos and discord; ChatGPT, however, does not suggest this resolution.
Ultimately, individual readers are the only ones who can satisfactorily "answer" the questions that a literary text raises. Literary analysis is a complex puzzle, and it requires nuanced and creative thinking to make sense of it—two traits that are beyond the capability of any computer program, no matter how sophisticated. ChatGPT can help generate interesting thoughts and ideas, but it is not able to accomplish the difficult task of interpreting literature in a way that satisfies a close reading of the text. In many ways ChatGPT produces a literary response that resembles a Modernist painting like Pablo Picasso's Girl with a Mandolin (1910):
We understand what Picasso is representing in an abstract sense, but the images are purposely jumbled and disconnected. By accumulating and combining all the various interpretations (and misinterpretations) of a text that have been taken from the internet and compiled in its vast dataset, ChatGPT's final analysis ultimately becomes a hodge-podge of conflicting ideas and theories that might produce interesting thoughts in isolation, but those ideas will need to be clarified, expanded, and organized before they can produce a coherent, consistent, and compelling argument. In other words, a proper literary analysis worthy of an Honors-level English class still requires a thoughtful, engaged student to write it.
In our classes, before students write any essay, we always have them share their preliminary ideas and observations in small groups and as a whole class. Knowing that "literature is the question minus the answer," students should seek as many differing opinions as they can. This investigatory process can certainly include ChatGPT as well. Perhaps there is an interesting computer-generated idea can be integrated into their own unique analysis. Ultimately, though, students need to construct their own arguments independently and support their claims with evidence from the text.
If students wanted ChatGPT to write a full essay in response to the prompt for Arnold's poem, the computer-generated output would look something like the following:
In Matthew Arnold's poem "Dover Beach," the speaker's observations of the external world reflect the complexity of his internal thoughts and feelings. Through his descriptions of the natural world, the speaker reveals a deep sense of sadness and disillusionment, as well as a longing for comfort and stability.
The speaker describes the sea as being "grating" and "dismal," reflecting the turmoil and confusion he feels within himself. The imagery of the sea receding, leaving the shore "bare," is symbolic of the loss of faith and the decline of religious belief that the speaker is experiencing. The speaker's description of the sea as being "dark and quiet" reflects his sense of isolation and loneliness, as well as his longing for comfort and peace.
The speaker's observations of the external world are also reflective of his sense of sadness and disillusionment. The "eternal note of sadness" in the sea's "tremulous cadence" is a powerful metaphor for the speaker's own feelings of sorrow and despair. The reference to the "ignorant armies" that "clash by night" is a powerful commentary on the violence and chaos of the world, which only serves to further heighten the speaker's sense of disillusionment.
Despite the speaker's sense of sadness and disillusionment, the poem is also a powerful expression of hope and comfort. The speaker's descriptions of the moonlight, which "gleams and is gone," and the stars, which are "eternal lights," are a powerful expression of the speaker's longing for comfort and stability in a world that is constantly changing. The speaker's longing for love and human relationships is also evident, as he recognizes that these are the only things that remain constant in a world that is in decline.
In conclusion, Matthew Arnold's poem "Dover Beach" is a powerful exploration of the speaker's internal thoughts and feelings. Through his observations of the external world, the speaker reveals a deep sense of sadness and disillusionment, as well as a longing for comfort and stability. The speaker's descriptions of the natural world are a powerful expression of the complexity of his internal thoughts and feelings, and offer a powerful commentary on the changing nature of the world and the role that love and human relationships can play in providing meaning and purpose in a world that is in decline.
Again, our first reaction when seeing what the computer can instantaneously generate is amazement; however, upon closer inspection this essay proves to be a superficial analysis that is riddled with holes and inconsistencies. The argument in the opening paragraph claims that the speaker desires "comfort and stability" in a world filled with "sadness and disillusionment." While that overall argument is interesting, the first body paragraph delves immediately into the "grating [roar]" of the crashing waves in the second half of the opening stanza without ever mentioning the contrasting beauty that is located in the first half.
The computer also claims that the poem is about a "world that is in decline" without ever mentioning "the eternal note of sadness" (14) that Sophocles' also heard "on the Aegean" (16) centuries ago that "brought / Into his mind the turbid ebb and flow / Of human misery" (16-18), suggesting that these feelings transcend time and place and reflect a fundamental aspect of the human condition. Overall, the thoughts and ideas generated by ChatGPT are certainly thought-provoking and could be used by students to help formulate their own arguments, but if they were to submit this essay as their final revision, it would receive a less-than-satisfactory grade in any college-preparatory English class.
When it comes time for students write their own first draft in our classes, we simulate what they will one day be required to do on the AP Literature Exam: namely, use pen-and-paper to construct a written argument within a limited time-frame (i.e. roughly 40 minutes). We spend multiple days preparing to write the essay, but when it comes time to put their thoughts on paper, it becomes an authentic performance task without outside assistance—meaning they cannot use their notes or a computer.
Ultimately, our goal is to prepare students for college-level writing, which requires that they master the skills and techniques necessary to succeed not only on the AP Literature Exam—which will present prompts, poems, and passages that they most likely have never seen before—but also on the notorious blue-book exams that they will face in a typical college-level English course.
After students have written their initial drafts, they will work in small groups to provide peer feedback. We first go over an anonymous sample essay as a whole class that has model questions written in the margins that I would ask if I were a peer responder for this particular essay:
We instruct students to use the Socratic Method when giving feedback, meaning that they should only ask clarifying or probing questions in the margins. They should not make any declarative statements on what the writer should or should not do; instead, they should ask the questions that come to their minds whenever they are confused (i.e. clarifying questions) or whenever they want the writer to think differently or more deeply about the subject they are discussing (i.e. probing questions). Oftentimes, asking probing questions is simply playing devil's advocate to get the writer to think about possible counter-arguments to their claims.
After we read the anonymous essay aloud and discuss the model questions I have written in the margins, students are asked to give a score for what they think the draft deserves. I prefer the simplicity of the six-point ACT Writing Rubric rather than the more complicated AP Scoring Rubric, but we go over the criteria of both so students understand what each score represents. Basically, both rubrics can be reduced to the following six-point holistic scale:
To add a greater degree of nuance to student evaluations, I allow peer responders to add a "+" or "-" after the score to indicate subtle differences in their overall evaluations. After we have gone over how we would assess the sample essay, students work in small groups to provide peer feedback and evaluate their classmates' drafts. The first peer responder asks questions in the left-hand margin (one for each paragraph) and the second peer responder asks questions in the right-hand margin. Both peer responders also provide a final evaluation next to their names at the top of the margin based on the six-point scale.
When students have finished peer responding, they will then return the essays to the original writer who will read the questions that have been posed in the margins. Groups will then turn their desks to face each other and discuss each essay, one at a time, to ensure that every student understands the feedback and knows what to focus on during the revision process. After the discussion, students will also provide a self-assessment at the top of the page based on the same six-point scale. After the peer response and small-group discussions, the student's essay should look like the following:
With this handwritten draft as a baseline, students should then work on a revision to craft as strong an argument as possible. If students want to use ChatGPT for feedback, they can type up their essay and submit it with a request for how it might be improved. When the above essay was typed out and submitted to ChatGPT, this was the feedback that it supplied:
The essay provides a good analysis of two poems from the Harlem Renaissance, "The White House" by Claude McKay and "A Black Man Talks of Reaping" by Arna Bontemps. Here are a few ways you can improve the essay:
1. Start with an introduction that sets the context for the essay, such as a brief overview of the Harlem Renaissance and why it was significant for African Americans.
2. Use quotes from the poems to support your analysis, but make sure to explain their significance.
3. Provide more concrete examples from the poems to support your claims. For instance, you could discuss specific images or themes in each poem that convey the message of the poem.
4. Consider using literary terms, such as tone and symbolism, to help clarify your analysis.
5. Add a conclusion that ties together the themes you discussed in the essay and reflects on the overall significance of the poems in the context of the Harlem Renaissance.
6. Revise for clarity, organization, and grammar. Consider breaking up longer paragraphs into shorter ones, using clear and concise language, and making sure the essay has a strong overall structure.
While the computer-generated feedback is somewhat generic, it does provide some helpful guidance, such as giving more historical context to the significance of the Harlem Renaissance in the opening paragraph. The feedback does not compare, however, with the specificity and depth of thought that will be provided by the peer-response questions, the small-group discussions that follow, or the teacher feedback that is provided in the next step of the revision process.
If students want, they can also request that ChatGPT rewrite their entire essay to improve it, which might look like the following:
The Harlem Renaissance was a challenging era for African Americans, marked by widespread discrimination and prejudice from white people. However, black artists used music and literature to express their experiences and emotions, inspiring others with their powerful works.
Two notable poems from this era are "The White House" by Claude McKay and "A Black Man Talks of Reaping" by Arna Bontemps. "The White House" reflects the frustration and anger felt by many black people at the time, as evidenced by lines like "Your door is shut against my tightened face, and I am sharp as steel with discontent." The poem also touches on the fear of white hatred, as expressed in the line "Oh, I must keep my heart inviolate – Against the potent poison of your hate."
On the other hand, "A Black Man Talks of Reaping" focuses on the unfairness experienced by the black community, as seen in the line "yet what I sowed and what the orchard yields – my brother’s sons are gathering stalk and root." The poem also highlights the struggle of black people to provide for their families and the bitter fruits they are left with, as seen in the line "Then my children glean in fields they have not sown, and feed on bitter fruit."
Despite the difficulties faced by black people during the Harlem Renaissance, both poems also convey hope and optimism. In "The White House," the writer states "But I possess the courage and the grace – to bear my anger proudly and unbent," and in "A Black Man Talks of Reaping," the line "I planted safe against this stark lean year" suggests a belief in a better future. Both poems demonstrate the resilience and determination of black people to overcome adversity and succeed.
Once again, ChatGPT offers some helpful feedback that could be used by students to improve their essays. For instance, changing the student's description of the Harlem Renaissance in the opening sentence from a "tough time" to a "challenging era" is certainly a more sophisticated way of describing the time period. However, ChatGPT makes some curious editorial decisions as well, such as eliminating from the introductory paragraph the two poems that will be compared and contrasted in the body of the essay. When students use ChatGPT's recommendations to improve their essays, they should be careful not to think all the "improvements" that the computer suggests will necessarily make their essays better.
In the long run, teachers and students should just see ChatGPT as another tool that students can use to generate ideas and receive feedback on their writing. Ultimately, though, students will still need to do the hard work of independently organizing their thoughts and constructing their arguments to produce a quality essay. Fortunately—for teachers and students alike—there are no shortcuts in the difficult task of literary analysis because the value in the process is having to wrestle with the inconsistencies and ambiguities that are defining characteristics of all great literature.