Another look at Indiana voucher study

Much of initial reporting on a groundbreaking study of Indiana’s school voucher program, including mine, suggested that voucher students do OK academically if they stay in private schools for four years. But a closer look raises questions about that narrative.

The study’s headline finding is that voucher students, on average, fall significantly behind their public-school peers in math performance while faring about the same in English/language arts. Given what we know, that’s really the message policymakers and the public should take from the research.

The study, by Joe Waddington of the University of Kentucky and Mark Berends of the University of Notre Dame, was released Monday. Its findings were covered by National Public RadioChalkbeatEducation Week, the Indianapolis Star and the Fort Wayne Journal-Gazette. A headline in the Washington Post was typical: “School voucher recipients lose ground at first, then catch up to peers, studies find.”

But the students who “catch up” are only a handful among voucher students included in the study. The study analyzed test scores for 3,913 students who received vouchers during the first four years of the Indiana program, from 2011-12 to 2014-15. But they had four years of test-score data for only about 5 percent of those students.

The only students to produce four years of data were those who were in certain grades and received a voucher in the first year of the program. (Indiana gives standardized tests to students in grades 3-8, so those who were in sixth grade or higher in the first year of the program would have aged out of the annual tests in four years).

By contrast, the researchers analyzed two or more years of data for over half the students in the study. So the finding that voucher students lost ground in the first and second year after moving to a private school seems considerably more solid.

The study also includes data for 15 percent of voucher students who left their private school and returned to a public school. Those students, on average, had fallen further behind in math than the typical voucher student, and they also had lost ground in English/language arts.

Here’s a possible interpretation: Maybe the voucher students who stayed in private schools for four years weren’t representative of all low-income students who received vouchers. Maybe they were disproportionately high-achieving or ambitious students who were more likely to make a go of it in private school.

And the students who gave up or lost their vouchers and returned to public schools – maybe they left because their parents saw the private school wasn’t helping them and may have been hurting. Or maybe the private schools, which can set their enrollment standards, gave some of those students a nudge.

Maybe the message isn’t that voucher students who stick with private schools do OK academically, but that voucher students who do OK academically are more likely to stick with private schools.

If that’s the case, you can’t make a credible claim that voucher students will necessarily regain what they lost if they just persist in their private school. Some will, but others won’t.

I asked the researchers if it made sense that voucher students are self-selecting to stay in or leave private schools on the basis of whether they’re successful, and Waddington said that may be the case.

“We don’t know the baseline differences between the eventual stayers and exiters as of right now, but we plan to dig into this,” he said by email. He added that he, Berends and another colleague are also working on a paper that will examine whether private schools are “pushing out” some voucher students.

There are just so many questions about Indiana’s voucher program. And as Superintendent of Public Instruction Jennifer McCormick has pointed out, we’re spending $146 million on vouchers without knowing the answers.

Meanwhile, the headline result from the study should stand: Students who receive vouchers, on average, are likely to lose ground when it comes to learning math.


1 thought on “Another look at Indiana voucher study

  1. Pingback: Indiana Voucher Study: Students Fall Behind, But Do They Really Catch Up After Four Years? | Diane Ravitch's blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s