Big, impressive study, questionable policy conclusions

A study of the impact of teachers on student success has been drawing lots of attention, including a big story in the New York Times, praise from columnist Nicholas Kristof and analysis in the blogosphere.

On the one hand, the paper by economists Raj Chetty, John Friedman and Jonah Rockoff offers new evidence that good teaching has long-lasting and far-reaching effects. This suggests that the recruitment, preparation and support of teachers should be a high priority for the nation.

But the economists also use their findings to call for rating teachers on the basis of “value-added” models, which use complex formulas to measure teachers’ impact on student test scores – and for firing teachers who don’t measure up. Annie Lowrey writes in the Times:

The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.”

This is a little surprising, given that, in the study itself, they caution against that sort of policy conclusion.

“Overall, our study shows that great teachers create great value and that test score impacts are helpful in identifying such teachers. However, more work is needed to determine the best way to use VA for policy,” they write in the executive summary.

They add that two important questions must be resolved before value-added models are used to evaluate teachers. One is whether attaching high stakes to test scores will skew results so much that it undermines the accuracy of the models. The other has to do with the economic cost of firing teachers, sometimes in error – the very “mistakes” that Chetty said would be trivial.

Then the New York Times calls and they throw caution to the wind.

The study reportedly breaks new ground Continue reading

Advertisements