Effective schools aren’t the ones you expect

The most effective school districts in Indiana aren’t the affluent suburban districts that produce the highest test scores year after year. Instead many are smallish, rural districts that don’t get much attention outside of their own communities.

That’s according to data from Stanford professor Sean Reardon, whose research shows how well the nation’s school districts did at improving test scores over a five-year period.

The results “defy how we typically think about wealth and education in America,” the New York Times said in a report on Reardon’s work. Students in affluent districts do tend to score higher on standardized tests; but when it comes to year-to-year growth, schools doing a good job are nearly as likely to serve many poor students.

“Many districts with high average test scores have low growth rates, and vice versa,” Reardon writes in a working paper. “And many low-income districts have above average growth rates.”

In Indiana, the standouts include Elkhart, Brownsburg, Middlebury, South Vermillion, Blackford County, Southwest (Sullivan County), Speedway, South Adams and Eastern Greene. Other than exemplary student growth, there’s nothing obvious that those districts have in common.

Most districts show average growth – students move up by about five grade levels over five years. There are low-growth districts, including high-poverty districts like Gary, East Chicago and Kokomo but also some that are demographically similar to those at the top.

In fact, you would expect students in poor districts to have lower growth because students who struggle with poverty and other challenges are likely have a harder time learning. But the statistical relationship between test-score growth and district wealth is relatively weak.

Reardon calculated the change in scores between third-graders in the spring of 2009 and eighth-graders in the spring of 2014, the same cohort. In the most effective Indiana districts, students grew by 1.2 grade levels per year, making six years of progress in the five-year period.

The research, and the fact that it was featured in the New York Times, provides a welcome antidote to the myth that “good” schools serve upper-middle-class families and produce high test scores. If learning is what matters, we should look to growth, not performance.

Here is an Excel file that includes average annual growth for Indiana school districts, ranked from highest to lowest. (For five-year growth rates, enter the district you’re interested in to the “find a school district” box at the bottom of the Times story. The five-year rates are slightly more accurate because of rounding).

At the national level, the big surprise was the much-maligned Chicago Public Schools district, where students grew, on average, by six grade levels over five years. Reardon’s Stanford Center for Education Policy Analysis has a report focused on Chicago.

Another surprise is Tennessee. Much of the state had above-average growth, while most of the South lagged in growth as well as performance. Florida, a hotbed of “reform” under former Gov. Jeb Bush, had mostly below-average growth. (See Figure 3 in Reardon’s working paper).

As usual, here are some caveats:

  • These data are for one cohort of students over one five-year period. Would the results be different if, for example, we looked at students who were third-graders in 2012 and eighth-graders in 2017? Especially in small districts, chances are they might be.
  • If a lot of students moved into or out of a district between 2009 and 2014, that could have affected the findings. Maybe Chicago’s gains were influenced by the fact that, as the Chicago Reporter found, thousands of students left predominantly poor and African-American CPS schools for Gary and other cities. Reardon and a co-author looked at outmigration in the Chicago paper, however, and concluded it shouldn’t be a big factor.
  • Charter-school students aren’t separated but are included with districts in which the schools are located. In Indiana, that won’t make much difference except in charter-heavy Gary and Indianapolis Public Schools. There, you can’t tell if charters help or hurt.
  • As Reardon cautions, even test-score growth is a flawed measure of school quality. “If you’re a parent, you want your kid to be a lot more things than good at math and reading,” he tells Slate. “You want them to be creative, and be able to work with others well, and be happy, and make friends well, and have critical-thinking skills.”

Caveats aside, these results should cast serious doubt on efforts to evaluate schools on test scores and not growth. As Reardon writes, the myth that wealthy schools are better doesn’t just mislead parents and the public – it can also create a market for high-status schools, “increasing economic segregation without improving school systems.”

Note: Thanks to Sean Reardon for providing the data that are the basis for this post. For those who are interested, here’s a downloadable spreadsheet with district-by-district data for Indiana, including average third-grade test scores, annual test-score growth, median income and results broken down for racial and ethnic subgroups.

2 thoughts on “Effective schools aren’t the ones you expect

  1. This is very interesting–thank you, Steve! I also strongly appreciate the cautions about using standardized tests of math and reading as a proxy for school quality, though I have no doubt that the teachers in the districts you highlight are doing excellent work with the resources they are given. I’d expand on what Reardon says. Parents are interested in whether their kids are happy and creative, yes. They also care about the types and breadth of courses offered, and the infrastructure. Do the schools have science labs and orchestras? Are there advanced courses? Are there a number of foreign languages offered with live teachers? Are the buildings and grounds in good shape? All of our students in all neighborhoods deserve these things, and they require money. It was really sad to hear our state superintendent say that in many Indiana high schools, the language teacher is Rosetta Stone.

  2. I agree this is a very interesting piece and the line of research presented is very important. However, schools ultimately have to prepare their students for whatever they are planning to do next: work, higher education, etc. And while “growth” is important, so too are results, and depending on how one measures them, the districts normally considered most effective can claim better ones generally.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s