Charter school study: ‘Tiresome’ questions remain

More than three-fourths of Indiana charter schools perform worse or at least no better than local public schools, according to a study released last month by Stanford University researchers.

That’s one way of spinning the results of the study from the Center for Research on Education Outcomes, or CREDO – although it’s not the finding that was played up in most news coverage. Nevertheless, there it is:

// In math, 42 percent of charter schools performed worse than traditional public schools, 23 percent performed better, and 35 percent were not statistically different.

// In English, 18 percent of charter schools performed better than traditional public schools, 8 percent performed worse and 74 percent were not statistically different.

Most media reports focused on the study’s finding that charter students, on average, scored better on tests than they would have if they attended their local public schools; or its conclusion that Indiana charter schools are among the best in the nation at improving their students’ test scores.

An Indianapolis Star editorial even argued the study should put an end to the “tiresome debates” over the effectiveness of charter schools. It won’t, and it shouldn’t.

A few issues that deserve continued exploration:

// Are charter-school students representative of the general population? The Star highlighted the surprising finding that Indiana charter schools serve significantly more poor and minority students than traditional public schools. CREDO reaches this conclusion (see Table 1, page 12) by comparing charter students with students in “feeder schools,” which include all public schools where at least one student attends a charter school. CREDO Director Macke Raymond, the author of the study, very patiently tried to explain to me, by phone, why this approach makes sense from an economics standpoint.

But there are charter schools all across Indiana, so the profile of feeder schools whose students attend charters looks similar to the state as a whole. Most Indiana charter schools, however, are in high-poverty urban areas like Indianapolis, Fort Wayne, South Bend and the Calumet region. So a non-economist might expect charter students to look like public-school students in those cities.
In fact, if you compare urban charter schools and public schools, it appears the charters enroll about the same percentage of poor students, more black students but fewer Hispanic and special-needs students. The profiles are similar, but they’re also different in interesting ways.

// CREDO finds that, on average, charter students did “significantly” better on ISTEP-Plus exams and End of Course Assessments than they would have done in non-charter schools. But “significantly better” doesn’t mean “a lot better.” It means enough better that it’s not a statistical error. The latest study follows up on a 2011 CREDO evaluation, which found similar test-score gains for Indiana charter students. Jonathan Plucker, who was then the director of the Center for Evaluation and Education Policy at Indiana University, told School Matters the gap was nothing to crow about. “From my perspective, they found no difference between the two types of schools,” he said.

// For some people, the take-home message from the CREDO study was that charter students are as much as two months ahead of their non-charter peers. Stanford led with that point in its news release. But the study itself says translating test-score differences into months of learning gains “is challenging and can be done only imprecisely,” and it warns that such results “should be interpreted cautiously.”

// The oft-made argument that charter schools may be “creaming” the best students doesn’t require thinking that charters exclude poor, minority or special-needs students, or that they dump or “counsel out” students who don’t make the grade. It’s at least arguable that the rigmarole involved in getting and keeping a child in a charter school – selecting the best fit, entering a lottery, arranging transportation – could deter all but the most motivated and savvy parents. And that those parents are more likely to provide their children with the support they need to do well in school.

// Comparing the charter and public students who qualify for free or reduced-price lunches or who receive special education services doesn’t really tell us too much. But that’s the only measure of poverty that’s available for studies like this one. As Chicago education researchers have shown, it can be much more challenging to improve schools with large numbers of truly disadvantaged students. Also, some studies have found that charter schools tend to enroll special-needs students with less serious disabilities, such as speech and language impairment. It would be interesting to know if that’s the case in Indiana.

CREDO has done these studies for 23 states, and Indiana charters are fifth most effective, according to center director Raymond. CREDO is best known for a 2009 national study of charter schools that’s often cited by both supporters and critics of charters. As Matthew Di Carlo of the Shanker Institute noted, the differences that CREDO finds between charter and non-charter schools tend to be small. You can’t really draw broad conclusions that “charters are good” or “charters are bad.”

Note: Apologies for posting this so late. The CREDO website was down over the holidays, so there was no way to have a live link to the study. I didn’t expect that from Stanford, but maybe even computers need a holiday.


5 thoughts on “Charter school study: ‘Tiresome’ questions remain

  1. The only way you could show that charters are significantly better than traditional public schools is to have a charter assume control over an existing public school and guarantee the student body stayed intact (no transfers in or out).

    If after five years, student showed consistent and significant growth, then maybe we would be onto something.

    Now the important question would be why students improved. What daily instructional approaches did the charter teachers use that were effective and could be implemented in other schools?

    In the final analysis, charters will not be better because they are called charters, pay teachers less, forbid unions, or make kids drill for tests.

    They will be better because they employ better instructional techniques, collaborate more effectively, or use a quality curriculum.

  2. Charters aso could turn out to be better because they are slightly better funded, because they can never be overcrowded, or because they are released from some of the regulations the state imposes on “ordinary” public schools. I think those issues are probably incredibly significant.

    Personally, as parent of one BPS child (who has an IEP — which is the reason we wanted him at BPS) and two MCCSC children, one of whom is in ALPS and one of whom still recieves a bit of speech therapy for some pronounciation errors but otherwise has no special needs, I’d be interested in your analysis of our local Bloomington options.

    My personal experiences are that BPS is a far better place for my particular autistic child than he last school became, but that his MCCSC elementary was a wonderful place for him until 5th grade.

    I have also seen that ALPS is a wonderful, wonderful place — if it is a good fit for your child, but that the logistics of it can nonetheess be very stupid. (Tell me again why we never even considered moving ALPS to the new Fairview building? Which is centrally located?)

    Personally, what I’ve seen as a individual, is that smaller systems can be more nimble and flexible. (They can also have fewer options from which to choose, of course.) I like the instances where the MCCSC — which I generally trust — can make its own decisions instead of having them imposed by the state or federal governments. And now, as a new BPS parent, I see that this flexibility travels well into an even smaller system. A system the size of just one school.

    • Vicky, thanks for posting this. I hope I would never question the decisions that any parents make about where there children will thrive. From a policy perspective, my beef with charter schools is that they seem likely to attract the very parents and children who could be effective at making the public schools better. (In Bloomington, I’m thinking of Fairview, as I often am when I write about local issues). I think the original idea and justification for charter schools was that, free from regulations, able to be flexible, etc., they would “innovate” and come up with approaches to teaching and learning that would benefit other schools. In practice, it seems like they’ve become an alternative that’s there for some students, whose parents are savvy and engaged enough to take advantage — which implies there will be many other kids who, through no fault of their own, don’t get that benefit (if that’s what it is).

      I realize, as I write this, that I’m thinking mostly about elementary schools, where, in my experience (from years ago!) it’s possible to be engaged as a parent and feel a sense of ownership and effectiveness in a public school. That certainly got a lot harder when my kids got to middle school and high school.

      As for why there wasn’t consideration of moving ALPS to Fairview — did that come up when you were on the board? Moving it from University or starting a second program there? I never heard about it being discussed. I would have my suspicions about why the idea might not have gone very far … What’s your take on why it didn’t work out at Summit?

      • That’s a lot to reply to!

        First of all, asking you to analyze our local options isn’t a suggestion that you second guess parents. That’s certainly not what I was thinking of anyway. All the BPS parents I know seem very happy with BPS and even if they read your blog I doubt you’d change their minds. 😉

        But people have to make decisions about their children’s futures, and when I was looking at BPS the only resource I had for that was BPS admins and parents.

        I have been an advocate of school choice for a long time, since before I got on the School Board, but I never managed to get anywhere advocating for it, even as a Trustee. I might have simply been ineffective, or it might have been a time of far too much chaos, what with us having 3 Superintendents in 2 years. I think school choice within the MCCSC would enhance our schools. I hear it’s been fantastic for Fairview to have a special program and a lottery. I think we could add a big, well-advertised school choice option to the “big” high schools and move to the rest of our schools as we saw the benefits.

        I mention this because you say you think charters attract all the really engaged and involved parents, but I don’t think that’s true. I think that Summit had lots of engaged parents when I was very involved there, but that we were quite limited in what we could really add or suggest. I believe that’s a systemic issue for the district, not a problem with the principal or the school or the PTO. There is so much state and federal red tape, the parents don’t (and shouldn’t have to try to) understand it, and the fund raising needs that are left are so obvious that they take most of the parents time. A PTO-run yearly science night is wonderful, I always go and my kids learn at them, but that’s not going to change the underlying culture of a school, or anything else dramatic like that.

        Moving the ALPS program to Fairview was, if I recall correctly, never so much as whispered while the new Fairview was being built. It didn’t occur to me at all, probably because I wasn’t an ALPS parent at the time. But I find the geographical inconvenience of the program annoying, and I understand that plenty of families on the southern side of the county find it so inconvenient that they simply don’t bother accepting a place.

        As for why it withered at Summit, my first guess — now in my 2nd year as an ALPS parent — is that the program is too specialized. It isn’t a broad or generic gifted and talented program. It’s a reading, writing and literature program for gifted and talented children. And for my own daughter it couldn’t fit her more closely or well. She loves it, it’s perfect for her, and if I could pry her out of the 4th novel she started this week I could probably get a good quote from her for you about it. 😉

        But why on EARTH don’t we have a similar program for g/t elementary kids whose talents are more in the math or science arena? Perhaps, as strange as it seems, there really are only 25 or so 4th graders a year for whom our current literature-oriented ALPS class sounds like a good risk, and geographic concerns aren’t large enough to add another classroom full.

        Thanks for asking. 🙂

  3. I would think that one way to find out if charter schools are improving/keeping the status quo/decreasing educational success is to compare individual student scores from when they attended public schools vs. their score form the charter school. If in fact the charter schools are getting a higher performing student, who may be poor or on the free and reduced lunch schedule, it won’t matter in the study because you will be comparing apples to apples not apples to oranges.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s