CREDO report boosts Indy charter schools

Indianapolis charter schools got a vote of confidence from a recent report by Stanford’s Center for Research on Education Outcomes, better known as CREDO. The study concluded urban charter schools are outperforming neighboring public schools, and Indy charters are doing better than most.

“It confirms a lot of the results we’re seeing on the ground,” said Brandon Brown, director of charter schools for the Indianapolis mayor’s office. “If you look across the state, the performance of charter schools is mixed. But if you look specifically at Indianapolis charter schools, they tend to consistently outperform traditional public schools.”

CREDO has critics. Some say it exaggerates the difference in performance between charter schools and public schools*. Others question its methodology, which compares charter students to statistically constructed “virtual twins” in public schools. There’s also concern that CREDO’s approach distracts from what makes schools effective and contributes to the “charter wars” – a zero-sum battle for reputation and students.

But the studies carry a lot of cachet and typically get a lot of press coverage. The center and its director, Macke Raymond, have been churning out detailed reports on charter schools for years. They have a giant database of student records and use a methodology that’s complex and hard to second-guess.

In the latest study, CREDO looked at charter schools in 41 urban areas from 2006 to 2011 and concluded that, in many cities, charters are doing a better job of boosting test scores than nearby public schools serving similar students. The study says that, overall, urban charter schools are providing students with the equivalent of 40 days of additional learning in math and 28 days in reading.

Indianapolis charter schools do even better at outpacing nearby district schools, according to the report – although Raymond cautions against making claims for additional days of learning at the city level. They do especially well for African-American students and poor students but don’t seem to provide much if any advantage for Latino and white students.

It has been widely reported that Indiana charter schools perform no better and often worse than public schools serving similar students. What’s different in Indianapolis? Brown attributes the success to the approach taken by the mayor’s office, which authorizes 36 of the city’s charter schools. It rejects four out of five charter applications, holds schools accountable to annual performance goals and shuts them down if they don’t succeed.

“We believe the results for charter schools in Marion County are a direct result of very rigorous authorizing,” Brown said.

According to the CREDO study, Indianapolis was one of 26 cities where charter schools performed better than local district schools. Charter schools did worse than district schools in 11 of the cities and about the same in four cities. The study said Indy is one of a few cities where urban students who remain in charter schools would be likely to catch up with the state average on test performance.

But some researchers question CREDO’s virtual twin approach and the center’s claim that it’s making apples-to-apples comparisons of students. Rutgers professor Bruce Baker ripped the latest report in a series of Twitter posts (starting March 20), concluding its findings were “junk.”

Jonathan Plucker, a University of Connecticut education professor and former director of the Center for Evaluation and Education Policy at Indiana University, repeated a criticism he has made of past CREDO studies. He said the performance differences CREDO claims are “minute by social science standards.”

On average, the center says urban charter students’ test-score growth is 1/20^th of a standard deviation greater than that of students in public schools. In Indianapolis, the charter advantage is 1/15^th of a standard deviation in math and 1/13^th in reading.

Most of us are clueless about standard deviations, so CREDO plays up converting the difference to additional days of learning provided by charter schools. But there are disagreements about that, too.

Matthew Di Carlo of the Shanker Institute notes another way to portray the same difference: If a public school student is at the 50^th percentile of all test-takers in math, a comparable charter student will be at the 52^nd percentile. That doesn’t sound as impressive as 40 additional days of learning.

But aside from the numbers, Di Carlo makes a bigger argument – that the “horse race” focus on whether charter schools or public schools are better is distracting us from the real question of which schools, charter or not, are doing a good job. And why.

“Schools’ performance varies by what they do,” he writes. “Not by what they are.”

* This post uses the term public schools to refer to public schools that are not charter schools. I’m not taking sides in the debate over whether charter schools are public schools. It’s just that alternative terms – e.g., non-charter public schools, traditional public schools or, God forbid, TPS schools – are clumsy or inaccurate. I use the terms here that parents and teachers typically use.