What Effect Does Professor Quality Have on Student Performance?

Whether or not (and how) professor quality affects student performance is an ongoing question in academia. Unfortunately, there are a lot of statistical problems involved in testing these effects. Here are a couple:

• Students self-select into professors classes. So, better students might choose to take harder professor, while weaker ones might choose "softball" ones. Or alternately, grade-conscious students might select into classes of "easy graders"
  
• Performance in a class is typically based on an exam that scored by the same professor teaching the class. This means that the professor can increase easily increase (or decrease) scores by "teaching to the test" or simply by inflating/deflating them directly
  
• Student evaluations are subject to a number of biases, including the physical attractiveness of the instructor (oh well, I'm screwed...). In addition, they're endogenous with respect to expected grades.
  

Here's a very interesting paper that addresses these problems, written by Scott Carroll (of UC-Davis) and James West (at the USAF Academy). The paper, titled "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors" uses some of the unique features of the class assignment system at the U.S. Air Force Academy to disentangle some of the confounding effects that make testing the professor quality/student performance so challenging. At the USAF, students are randomly assigned to class sections, faculty in common courses teach from a single common syllabus, and exams are jointly graded by all faculty teaching the course. Here's the abstract (note: italics are mine):

It is difficult to measure teaching quality at the post-secondary level because students typically self-select" their coursework and their professors. Despite this, student evaluations of professors are widely used in faculty promotion and tenure decisions. We exploit the random assignment of college students to professors in a large body of required coursework to examine how professor quality affects student achievement. Introductory course professors significantly affect student achievement in contemporaneous and follow-on related courses, but the effects are quite heterogeneous across subjects. Students of professors who as a group perform well in the initial mathematics course perform significantly worse in follow-on related math, science, and engineering courses. We find that the academic rank, teaching experience, and terminal degree status of mathematics and science professors are negatively correlated with contemporaneous student achievement, but positively related to follow-on course achievement. Across all subjects, student evaluations of professors are positive predictors of contemporaneous course achievement, but are poor predictors of follow-on course achievement.

So, "higher quality" professors result in better long-term performance for their students. But at the same time, their students get lower grades in the foundations courses. This could be a result of lower quality instructors "teaching to the test", or of better instructors teaching in a way that's geared more towards follow-on classes at the expense of focusing on the intro materials (those ar tow ways of saying the same thing, BTW).

If you don't have access to NBER, you can read an ungated version of the paper here.

HT: Kids Prefer Cheese