Indiana and other states should be doing more to detect schools that are cheating on standardized tests, according to a nationwide reporting project led by the Atlanta Journal-Constitution newspaper.
But education officials and at least one researcher have questioned the AJC’s methodology and conclusions, suggesting they focus suspicion on schools that probably haven’t done anything wrong.
The newspaper looked at test scores in schools across the country to try to identify anomalies like those that touched off an investigation of cheating in Atlanta schools. That probe found that at least 178 Atlanta educators tampered with tests.
AJC reporters flagged schools with year-to-year test scores changes that seemed too big to result from chance or good or bad teaching. For example, if test scores for a school’s fifth-graders in 2010 were markedly different from those of the same school’s fourth-graders in 2009, suspicions were aroused.
“The analysis doesn’t prove cheating,” the newspaper said. “But it reveals that test scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.”
According to the article, a school should expect to have up to 5 percent of its classes experience unusual changes in test scores from one year to the next. Districts that consistently have more than 10 percent of their classes flagged for unusual changes merit further scrutiny, it said.
But Western Michigan University researcher Gary Miron, writing for The Answer Sheet blog at the Washington Post, argues that the AJC’s methodology is flawed, and what may look like cheating, on further examination, probably isn’t.
Miron consulted with USA Today on its prize-winning investigation of possible cheating in Washington, D.C., schools, which looked at year-to-year changes in test scores for individual students as well as rates of erasures on the tests. He said the AJC’s approach – relying on class-level data, not individual scores – probably results in comparing test scores for different groups of students.
“My own analysis of the data suggests that these irregularities are less likely due to actual cheating than due to mobility in student population,” he writes.
The AJC provides a searchable data base that lets you see how many classes in your local school district were flagged. There’s also an interactive map. In Indiana, 22 districts had 10 percent or more of their classes flagged in at least one year; and they’re a mix: urban, suburban and rural, rich and poor.
The newspaper singles out Gary Public Schools for some of the biggest test-score discrepancies. It says the odds of a district having so many classes flagged were one in a trillion. But Gary, with its almost exclusively low-income population, fits the profile of a school that could be unfairly judged because of high student mobility. Gary schools spokeswoman Sarita Stevens questioned the data that the AJC used in its analysis, Northwestern Indiana’s Post-Tribune reported.
Alex Damron, spokesman for the Indiana Department of Education, played down the likelihood of cheating, pointing out correctly that Indiana has cracked down on test security in recent years.
“In Indiana, we have high academic and moral standards for our students and teachers,” he said. “We have every reason to believe incidents of cheating are the exception rather than the common practice.”
Damron told School Matters that a study of erasures on Indiana’s ISTEP-Plus exam, performed last year by the testing contractor at the state’s request, turned up “very few areas of concern statewide.”
If cheating were ubiquitous, it would undermine efforts to judge schools and evaluate teachers on the basis of test scores. But given the well-known risks of getting caught in a scandal like Atlanta’s – coupled with the states’ recent strengthening of test security – widespread cheating seems unlikely.
And as Miron argues, maybe we should be more concerned with the legal and socially acceptable cheating that comes with putting too much emphasis on standardized tests for math and English – cheating students out of a broad education that includes science, foreign languages, extracurricular activities and creativity.
His concluding line would make a good bumper sticker: “No cheating on tests is as serious as the cheating done by the tests.”