Interpreting Course Evaluations

How to Make the Best of Bad Course Evaluations

[Helpful ideas for interpreting student comments – course evaluations and non-formal feedback you may collect] 

Author: Manya Whitaker is an associate professor of education at Colorado College. 

Source: https://community.chronicle.com/news/2206-how-to-make-the-best-of-bad-c… evaluations

Let’s start by acknowledging the truth: Course evaluations are incredibly biased, and aren’t an accurate measure of an instructor’s effectiveness in the classroom. Too often, students’ perceptions of your appearance, demeanor, or pedigree prevent them from writing a fair and relevant review of your actual teaching. Yet despite dozens of studies demonstrating their unreliability, course evaluations continue to be used in hiring, tenure, and promotion decisions by most colleges and universities.

If you’re lucky enough to have tenure, you are no longer required to defend your teaching against specious anonymous comments. But for everyone else, your best bet might be to repurpose course evaluations from being solely summative to at least moderately formative. What I mean is: Your institution might use students’ ratings solely to judge your performance, but you can use them diagnostically. Mine the "data" for ideas on how to improve your class and your teaching.

Before course evals can become a useful professional tool, you need to learn a few rules of engagement:

• Don’t take the comments personally. Yes that can be hard to do if a student’s remark about your appearance or personality is particularly vicious. Some students do use these forms to attack their professors without consequence, yet other students offer insightful, appropriate comments on course organization, learning materials, assignments, quality and timeliness of feedback, and instructional methods. Focus on those, and put the others aside. If you don’t want to filter out disrespectful commentary, have a friend skim them first, and then send you the valid evaluations. 
• Take your students seriously. Don’t dismiss their opinions as ill-informed simply because they’re students. The fact is, while most faculty members are experts in their content, very few have any formal pedagogical training. No matter how well you know the material or how much effort you put into teaching, if students aren’t learning, something has to change. It could be something you’re doing, or something you’re not doing. Course evaluations can help identify what’s good, what’s lacking, and what needs a bit of work in your classroom. 
• Consider the results in context. Students routinely give lower ratings to required courses than to electives. The same is true of new courses versus established ones, as well as of introductory courses versus upper-level seminars. Read your course evals through the appropriate lens. 
• Don’t reread negative comments. Don’t keep going back to them. Just don’t.

Once you are certain you can follow the rules of engagement, grab a cup of coffee, and pen and paper, because you are about to engage in a bit of qualitative data analysis. What follows are some common remarks that show up on course evaluations. My goal here is to help you "decode" them and figure out the implications for your teaching.

"The class was too hard (too easy)." That kind of comment doesn’t tell you much about your course content, assignments, or tests; it mostly reflects how much time and effort a student devoted to the class. "Too hard" comments mean that they are spending what they perceive to be an above-average amount of time on your course. "Too easy" means they are accustomed to spending more time on coursework.

If you’re getting a lot of "too hard" or "too easy" comments, you can reduce their number by clearly articulating your expectations on the syllabus and at the beginning of the semester. Be explicit about how much time you expect students to devote to coursework and what they should be doing during that time. For example, I tell students that I expect them to go through each reading twice — once for content (i.e., purpose and key takeaways) and once for context (i.e., how it fits with other readings). I also articulate my learning outcomes ("by the end of this course, you will be able to …") so that students clearly understand what they should be working toward.

Instead of binary judgments of hard versus easy, here’s what you want to read on your course evals: "This class was challenging."

"The grading was unfair." Most of the time, such comments come from students who think they "deserved" a higher grade. The first thing to do is make sure your grading is indeed fair. For example, do you privilege one demonstration of knowledge or content over another? In my courses, all of the assignments are worth within 10 points of one another, with those completed earlier in the semester worth less than later ones. I also give a variety of assessments that tap into different skills — essays, research papers, oral exams, movie/product reviews, video blogs, podcasts, short documentaries, and so on.

Grade grubbing can result when students feel as if they are being asked to do something that they "just aren’t good at." To minimize that reaction, I use rubrics and provide examples of successful and unsuccessful assignments. When I have time in class, I ask students to score sample assignments using the provided rubric. Sometimes they just need to experience the difficulty of grading nonnumerical answers to better interpret their own academic outcomes.

The ideal grading comment: "The class was demanding, but I knew what I had to do."

"The assignments were confusing and pointless." Without having been a teacher, it can be hard to deduce the purpose of educational activities. After my third-year review yielded a few such comments, I began to explicitly link each assignment to course content. I explained the specific purpose of the assignment — to help practice research skills, to demonstrate your command of last week’s material, to identify how theories translate to practice, etc. When an assignment is complicated, I allot class time for students to create study plans to make sure they understand what is being asked of them, and why.

Nowadays the kind of comment I’m more likely to read on my course evaluations: "The assignments were each designed with specific goals in mind."

"He/she spent too much (not enough) time on X." When you read that comment on your evals, it means your students have a fundamental misunderstanding of the purpose and goals of your course. There are three things you can do to head off such remarks:

On Day 1, spend some time talking about what your course is and what it’s not — or, put another way, the boundaries of the course. Share your disciplinary orientation. That offers students a rationale for why you will be focusing on some topics more than others. Finally, tell students how your class fits within the departmental curriculum. It could be that the No. 1 thing a student wants to study is the central concept of a different course.

Five minutes of explanation can go a long way toward producing the comment you’d rather see on your evals: "The class matched the course description."

"The professor played favorites." It is tempting to read that as a personal attack. More likely, however, such criticism reflects classroom dynamics. Think about how you organize group projects or call on students in class. Consider whom you chat with casually before and after class. Do you regularly make eye contact with certain students, and not with others?

The reality is that faculty members do have favorite students, but that should not factor into our teaching practices. One way to check those biases is to grade blind. Ask students to use their ID numbers instead of their names when they submit work. This will keep you from subconsciously raising or lowering expectations for certain students. 

Goal comment: "The professor made everyone feel comfortable sharing their opinions and asking questions."

"The professor/course was disorganized." For me, that type of comment is the most instructive. It tells me that I am not relaying my pedagogical intentions very effectively. Happily, there is an easy fix: Review your intentions in class throughout the semester.

On the first day of class, I explain the structure of the course and the curricular narrative. Every time we move to a new topic, I start with a review of prior content, often by having students create concept maps. I begin every class session with an overview/agenda for the day. I end class by revisiting the daily learning objectives.

I feel affirmed when I see this sort of comment on my course evals: "The material and class sessions were thoughtfully planned."

"This class was boring." Usually that comes from students who aren’t all that interested in the subject of your class, especially if they were required to take it. Then again, maybe your course is boring. Maybe there’s not enough activity, participation, or diversity in how you are presenting the material.

 Because my institution follows an unusual academic model, in which each class session is three hours long, I’ve been forced to get a little creative in my teaching. My No. 1 rule: Mix it up! For example:

• Throw away your PowerPoint slides and long lectures. They’re not as compelling as you think they are.
• Instead, draft 10-minute minilectures, and break them up with activities that students can do alone, in small groups, or as a whole class. Pick activities that operate in the middle and top levels of Bloom’s taxonomy — meaning the work requires students to make connections, apply their knowledge, or generate new ideas. In other words, have the students DO something in class. 
• Complement homework readings with nontraditional in-class texts — podcasts, videos, miniseries, documentaries, poems, music, and art. This has the dual benefit of presenting the material in a different way while providing students with something novel to grab their attention.

The comment I hope results from my own efforts to mix it up: "Even though I wasn’t initially interested in the course, I learned a lot."

All the logistics. Once you winnow out the personal attacks and the pedagogical comments, all that is left are complaints about course logistics. Chief among them: 

• The professor is not available outside of class (frequently reiterate your office hours and their purpose). 
• The professor doesn’t return papers and tests fast enough (just be sure to give them feedback before their next assignment is due).
• The texts cost too much (advise students to borrow or rent books, or to purchase e-books; or consider creating a course reader).

Fixing such snafus is easy. Just don’t let them slide. 

Let your teaching philosophy and pedagogical goals guide you in the effort to find something from course evals that you can use to improve your teaching. Do you care more about students learning the content or acquiring skills? Are you more interested in breadth or depth of knowledge? How do you want students to feel when they are in your classroom? 

The answers to those questions not only should guide your initial course design; they should also inform how you engage with course evaluations.