Scoring Essay Tests
Charles Champlin (2006), a journalist for Time and Life magazines, describes his experience of taking essay tests as a student at Harvard:
“The worst were the essay questions (which seemed only distantly related to whatever you’d read or heard in lectures). They made a statement and then simply said, ‘Discuss.’ O terrifying word, ‘Discuss.’ Nothing so simple as tossing in a few facts retained from all-night cramming. It was meaning that was sought – which was, as I’d already begun to appreciate, the way it should be. But it was a strained step up from the exams I’d known before, when memory, regurgitated, would get you around almost any corner.”
Champlin’s reminiscence reveals some of the strengths and dangers associated with essay questions. They are a wonderful way to test higher-level learning, but they require careful construction to maximize their assessment effectiveness.
I. Strengths Associated with Essay Examinations
Among the strengths of essay examinations, faculty who use them find they are a valuable means to measure higher-order learning and a wonderful way, when scored properly, to further student learning. Given these strengths, essay tests require careful preparation and scoring.
1. Essay Questions Test Higher-Level Learning Objectives
Unlike objective test items that are ideally suited for testing students’ broad knowledge of course content in a relatively short amount of time, essay questions are best suited for testing higher-level learning. By nature, they require longer time for students to think, organize and compose their answers.
In the table below, appropriate testing strategies are associated with Bloom’s hierarchy of learning. The action verbs under each domain illustrate the kinds of activities that a test item might assess. Use the verbs when constructing your essay questions so that students know what you expect as they write. While essay questions can assess all the cognitive domains, most educators suggest that due to the time required to answer them, essay questions should not be used if the same material can be assessed through a multiple-choice or objective item. Reserve your use of essay questions for testing higher-level learning that requires students to synthesize or evaluate information.
2. Essay Questions When Scored Properly Can Further Learning
Teachers score essay exams by either the holistic approach or the analytic approach.
The holistic approach involves the teacher reading all the responses to a given essay question and assigning a grade based on the overall quality of the response. Some teachers use a holistic approach by ranking students’ answers into groups of best answers, average answers and poor answers and subdividing the groups to assign grades.
Holistic scoring works best for essay questions that are open-ended and can produce a variety of acceptable answers.
Analytic scoring involves reading the essays for the essential parts of an ideal answer. In this case, you will need to make a list of the major elements that students should include in an answer. You will grade the essays based on how well students’ answers match the components of the model answer.
Whichever method, holistic or analytic, that you use to score the exam, you should write comments on the students’ papers to enhance their learning. Your comments will help students write better essays for future classes and reinforce what students know and need to learn. Your comments are also a good reminder for yourself if students come to you with questions about their grades.
II. Dangers to Consider When Giving and Grading Essay Examinations
1. Establish limits within the essay question
The example of Charles Champlin’s experience at Harvard where his teachers gave a statement and then simply said, ‘Discuss,’ shows a danger in using essay questions. Instructors should build limits into questions in order to save needless writing due to vague questions: “With some essay questions, students can feel like they have an infinite supply of lead to write a response on an indefinite number of pages about whatever they feel happy to write about. This can happen when the essay question is vague or open to numerous interpretations. Remember that effective essay questions provide students with an indication of the types of thinking and content to use in responding to the essay question” (Reiner, 2002).
Another good way to prevent students from spending excessive time on essays is to give them testing instructions on how long they should spend on test items. McKeachie (2002) gives the following advice: “As a rule of thumb I allow about 1 minute per item for multiple-choice or fill-in-the-blank items, 2 minutes per short-answer question requiring more than a sentence answer, 10 to 15 minutes for a limited essay question, and half-hour to an hour for a broader question requiring more than a page or two to answer.”
2. Remember that essays require more time to score
While essay exams are quicker to prepare than multiple-choice exams, essay exams take much longer to score. You should plan sufficient time for scoring the essays to prevent finding yourself crunched to report final grades.
3. Avoid scoring prejudices
Essay exams are subject to scoring prejudices. Reading all of an individual’s essays at the same time can cause either a positive or a negative bias on the part of the reader. If a student’s first essay is strong, the examiner might read the student’s remaining essays with a predisposition that they are also going to be strong. The reverse is also true. To prevent this scoring prejudice, educators suggest reading all the answers to a single essay question at one time.
Champlin, C. (2006). A life in writing: the story of an American journalist. Syracuse: Syracuse University.
McKeachie, W. (2002). McKeachie’s teaching tips (11th. ed.) New York: Houghton Mifflin.
Reiner, C., Bothell, T., Sudweeks, R., & Wood, B. (2002). Preparing effective essay questions. (http://testing.byu.edu/info/handbooks/WritingEffectiveEssayQuestions.pdf).
Candace Caraco, TA, Department of English
Grading student papers for a course in any discipline presents a series of challenges different from grading other kinds of assignments. Typically, a wide range of responses will be acceptable, and every paper (unless it is plagiarized) will have some merit. Consequently, grading essays demands a teacher’s close attention to insure that each paper is judged by the same standards. A method for evaluating essays that breaks the grading process into parts can help an instructor work more consistently and efficiently. By assessing papers based upon the three general categories of ideas, argument, and mechanics and style, categories easily adapted for each discipline and assignment, an instructor can more easily recognize and comment on an essay’s strengths and weaknesses and so face that daunting pile of twenty, forty, or even one hundred essays with less trepidation. Furthermore, if teachers make clear to students how this method works, fewer students will be confused about their grades or apt to charge that papers are graded in an arbitrary or purely subjective way.
Before applying the three categories for evaluation, think through what it is you want an assignment to accomplish. Grades should reflect the most significant strengths and weaknesses of an essay, so a teacher should carefully consider ahead of time what expectations he or she has for a paper and especially what he or she most wants students to do for a particular assignment. For example,
- Do the instructions to students require specific tasks, such as agreeing or disagreeing with an author, outlining a book’s argument for review, or analyzing a particular section of a work?
- What is it that students should show they understood?
More generally, a teacher may also consider the following:
- Has the student presented ideas in a logical order?
- Is the essay written in clear, grammatically correct prose?
- Has the student offered explanation or examples to support generalizations?
For any given assignment, your criteria for success may vary in the details; whatever they are, make a list of them. Ideally, students would receive a copy of this list before they begin writing their essays.
The problem with such a list of criteria, however, is that it can quickly grow unwieldy. While we need some specific questions as a checklist for student writing success, we can benefit from a streamlined evaluation system. The ideas/argument/mechanics and style format is a simple way to group criteria, both for yourself and your students. Once you have a set of criteria for an essay to succeed, you can decide how these questions fit under the three headings. A general breakdown of these questions might look like this:
- Does the student understand the accompanying reading or the principles behind the experiment, etc.?
- Does the student offer original interpretations?
- Do the student’s explanations of terms, ideas, and examples demonstrate an ability to grasp the main points, paraphrase them, and apply them?
- Does the student answer the question(s) assigned?
- Does the essay demonstrate an understanding of a subject, or does it wander from one subject to the next without offering more than superficial remarks?
- Can we easily determine what the author’s main point is?
- Does the essay provide a series of points that add up to an argument supporting the main point (thesis)?
- Does the essay proceed logically from point to point?
- Does the student provide examples and explanations to support his or her generalizations?
- Does the essay contain contradictions? Is the paragraph structure logical?
MECHANICS AND STYLE
- Is it clear what the student’s point of view is?
- Does the student control tone? Is the essay free of grammatical errors?
- Is the essay punctuated appropriately?
- Do citations and bibliography follow the correct format?
- Is the prose clear or do you puzzle over individual sentences?
- Are words spelled correctly?
What I am suggesting is essentially adapted from the methods of two English professors, Charlene Sedgwick and Steve Cushman. Sedgwick’s “ENWR Handbook” offers guidelines for evaluating freshman composition papers by assessing focus, organization, style, and mechanics; Cushman has in the past recommended that graders for his upper-level literature courses weigh mechanics and style (together) as one-third of a grade, and ideas and argument as the other two-thirds. Though instructors for non-English courses may want less emphasis on writing skills per se in an essay grade, I would argue that papers for all courses should be evaluated at least in part for their grammar, punctuation, and prose style because these fundamentals of writing are everywhere necessary for readers to understand writers. And a teacher in any discipline can easily tailor the three categories of ideas/argument/mechanics and style to the conventions of the course and its academic discipline.
Simplified (and Platonized) then, these three categories translate into the following grade scale: essays with good ideas that are logically organized into an argument and written in clear and mechanically clean prose receive an A; essays lacking in one category (e.g., have poor organization) receive a B; essays weak in two categories receive a C; and essays that manage none the three general criteria garner a D or fail. What constitutes an “A” within any given category will also depend upon the course level and the assignment, but in a very general way, if a student’s essay can answer “yes” to all of your questions for a category, then the student should have an “A” for that portion of the grade. (More explicit criteria appear in “Responding to Student Writing” by Stella Deen in the November, 1995, Teaching Concerns.)
Particularly for new teachers, it is sometimes helpful to read through several essays to see what an average paper for a class looks like. Checking to see if several papers have similar difficulties can also help you detect unclear instructions in the assignment or a content issue that may require further class discussion: if we have been unclear in some way, then we should be prepared to cut our students some slack when evaluating that part of the assignment.
However much we simplify the process, grading essays will never be as simple as marking multiple choice exams. Most student essays are some combination of good ideas and slight misunderstanding, clear argument and less clear argument: they don’t neatly divide into three parts. Typically the problems in an essay are closely related: for example, a misunderstanding of content can lead to a logical flaw in the argument and to prose that is full of short sentences because the author is not certain which ideas should be subordinated to others. Because of this system of logical relations, it is all the more important to include a final comment with a grade.
Writing final comments may indeed slow grading, but the pedagogical benefits of comments far outweigh the few minutes per paper needed to write them. Students continue to learn from an assignment if they understand what their work accomplished and what it didn’t. More importantly, final comments can help students write more fully conceived and better executed papers on the next assignment. (For a time-saving method of offering detailed comments about common problems in a set of essays, see Nancy Childress’s essay “Using General Comment Sheets,” published in the October, 1995 issue of Teaching Concerns; she recommends preparing a handout for the entire class in addition to [shorter] written comments on individual essays.)
One way to organize an end comment is to write at least one sentence pertaining to each of the three categories of ideas, argument, and style and mechanics. Breaking an essay into these three components can help us comment on an essay’s strengths and weaknesses more quickly than if we had no set criteria or if we had too many. A particularly successful comment will explain to a student how ideas, argument, style, and even grammar work together. Final comments also serve as a check on ourselves, especially if we tie our general end comments to specific examples within the paper. For example, when I finish reading Student A’s essay, I may sense that he didn’t offer proof in support of assertions. But when I look for an example of an unsupported assertion, I find there are passages that might serve as supporting evidence; however, he has not explained very carefully how the examples work, so my impression has been that his essay lacks proof. Even when we are sure that we have avoided bias and inconsistency, comments pointing to examples will better illustrate to students what they can improve. Above all, comments should not be mere justifications for grades, though they may coincidentally deter students from seeking explanations as to why the received a “B” instead of an “A.”
TRC NOTE: For help in implementing these suggestions, request a Writing Workshop.