John Ewing wrote a classic 2011 journal article, titled “Mathematical Intimidation,” that lamented the growing use of value-added models to evaluate schools and teachers. Three years later, he sees little evidence that education policy makers understand or care about the flaws in the approach.
Yes, he said, critics of value-added have grown more vocal. But lots of people with power and influence are still wedded to the idea that we can use test scores to identify bad teachers – and either weed them out of the profession or make them improve.
“People just can’t let it go,” Ewing told me this week. “Policy makers bought in, in a big way, and they can’t let go of it.”
Ewing, a mathematician, is president of Math for America, a New York-based organization that promotes mathematics education. He previously spent 14 years as executive director of the American Mathematical Society. Before that, he was an Indiana University math professor for two decades.
“Mathematical Intimidation” was directed at his fellow mathematicians, urging them to stand against policies that make bad use of their discipline. But it’s a concise, easy read. You don’t need to be a mathematician, or even know a lot of math, to follow its clear and persuasive argument.
Ewing wrote that that proponents of value-added use the supposed objectivity of the models – they’re based on mathematics, after all – to close off discussion of what the goals of education should be. But the models rest on a shaky foundation: The idea that standardized tests in math and English provide a valid and complete measure of what schools and teachers should accomplish.
A huge problem with using value-added to judge teachers is that studies have found VAM scores to be wildly unstable: A teacher who does great one year may have a lousy rating the next. “There’s lots of evidence, and the evidence is mounting,” Ewing said. “I have no doubt that, in a decade, it will be seen as an idea that didn’t make any sense.”
But in the meantime, the idea is doing a lot of harm. Witness the recent attacks on teacher tenure and due-process rights in California and New York, underpinned by the idea that we can identify the worst teachers with test scores. And laws and policies in a majority of states that require test-score-based teacher evaluations. And stories from across the country about administrators and teachers who have cheated to boost test scores.
“Mathematical Intimidation” describes “an episode reminiscent of the (Chinese) cultural revolution” in which reporters track down and confront teachers over low VAM scores. This happened in Los Angeles, where the LA Times published its own value-added evaluations for the city’s teachers; and more egregiously in New York, where tabloids hounded the city’s “worst teachers” as if they were criminals.
Math for America works with teachers, and Ewing told me he sees plenty of evidence that promising educators are discouraged not only by such disrespect but by being subjected to evaluation schemes that don’t make sense. He’s not saying we shouldn’t hold teachers accountable. They should be accountable, he said, for knowing their subject matter, knowing the craft of teaching and knowing the needs of their students. “And, by the way, you can measure each of these things,” he added.
But good teaching is complex and multi-faceted, and evaluations should take that into account, he said. We’re deluding ourselves if we think a single approach can be used to rate all schools, students and teachers, regardless of their differences, if only we use a super-sophisticated statistical model.
“The idea that you can make up for the differences with some mathematical formula is nuts,” Ewing said. “It’s a terrible misrepresentation of what mathematics can do.”