New school grading system produces little change

The new school grading system that Indiana will adopt in 2016 is supposed to give more weight to student growth on standardized tests and less to straight-up test performance, making it more likely that high-poverty schools can earn high grades.

But that may not happen. In a comparison of the grades that schools received in 2014 with the grades that they would have received if the new system had been in effect, there’s not much difference.

A majority of schools would have received the same grade under the new system as under the old. Almost no schools would have seen their scores rise or fall by more than one letter grade.

The Indiana Department of Education calculated grades that schools would have received, based on their 2014 test scores, if the proposed new system had been in place. The department provided the grades in spreadsheet format in response to a public records request.

According to my analysis of the data, over 60 percent of schools would get the same grade under the new system as they got under the old. Some 22 percent would drop one grade and 12.5 percent would rise one grade. Less than 3 percent of schools would rise or fall by two or more grades.

GradeChange4Among schools that received As in 2014 – many of them schools serving few poor children – two-thirds would get As under the new system. One-third would drop to Bs.

If there’s good news, it’s that quite a few schools at the bottom would do better under the new system – not a lot better, but a little. Among schools that got Fs, 45 percent would have raised their grades, most of them to Ds. Of schools that got Ds, 60 percent would have raised their grades, most of them to Cs.

But overall there wouldn’t be a lot of change, as this chart illustrates. (Each column represents the grades that schools received in 2014. The “layers” show the grades they would have received under the proposed new system).

GradeDistribution4If we’re going to engage in the dubious business of assigning letter grades to schools – as the Indiana legislature insists we must – it only makes sense to put more weight on student growth. As Matthew Di Carlo and others have pointed out, Indiana’s current performance-heavy system means grades correlate closely with student demographics. Schools that serve few poor students mostly get As. Schools that get Ds and Fs almost all serve lots of poor students.

While the data from the state don’t look promising, the new grading system isn’t a done deal, and the state board could still adopt an approach that leans more heavily to student growth.

Under the new system, students (and their schools) will be awarded growth points based on how much they improve their test scores as measured by a “growth to proficiency table.” Several such tables have been considered, and the board won’t approve the growth table until this fall.

The Department of Education provided the grades schools would have received under the first of these proposed tables (shown below), which seems to be the one that state officials favor. It awards considerably more growth points to students who pass ISTEP+ exams than to those who don’t t pass the tests.

600----14-447_Growth_to_Proficiancy_Table_incorporated_by_511_IAC_6_2-10-2

For example, students in the highest testing categories get 150 growth points if their scores show positive movement; students who didn’t pass get only 100 points for positive movement. And students who passed the test but don’t improve get 50 or 75 points, while those who didn’t pass and don’t improve get none.

That arguably turns the formula’s growth measure into a second performance measure, which could explain why most schools’ grades change little with the new system.

Advertisements

5 thoughts on “New school grading system produces little change

  1. You’re on a roll with these posts. The big question is will they use the peer referenced system that we have already argued against the legality of but officials have insisted on. If it is a peer referenced system than it should be no surprise that the grades are similar, but if it is truly criterion reference, than it would be impossible to predict ahead of time what the results would be (as it should be).

  2. Steve… I agree that you are doing great work on these analyses. Indeed, I am unaware of anyone doing similar work, which is really disappointing and disheartening. Yet, I think you (and others, including DOE officials) are shortsighted in judging the impact of these measures by running the new system on old performance data. Doing so assumes that different accountability systems will have the exact same impact on behavior; and yet, an accountability system is designed precisley to drive certain behaviors – and changing the components of an accountability system, if effective, should change the behaviors that the system incentivizes. Nonetheless, I think in the end your analysis may be correct – even if not for the right reasons. This new system will change very little. The primary culprit, I predict, will be the continued reliance on peer referenced systems, as noted in the post above by JF. This is not “growth to standard” as the law requires – and which schools should consider to be the most fair measure of their performance . Some may actually think that peer based measures are the only fair measure; and yet, for lots of interesting statistical reasons, the practical rollout of such measures has proven to be anything but fair. But alas… it is the perceptions of fairness, not real measures, that continue to drive these poorly designed systems.

    • Derek, thanks for the comment. I’d be interested in hearing your thoughts in more depth on this topic. On vacation this week and next and trying to give the blog a rest, but let’s be in touch later if you’ve got time and are willing.

  3. Pingback: How will Indiana factor growth in school grades? | School Matters

  4. Pingback: Board faces weighty decision on rewarding test-score growth | School Matters

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s