Nuanced findings in study of ‘learning loss’

Indiana students lost nearly six months of learning in math and over four months in reading as a result of the COVID-19 pandemic. That’s an obvious and accurate headline about the Education Recovery Scorecard, a study of COVID’s academic effects led by Stanford and Harvard researchers.

An alternative headline might be: It’s complicated. The findings, based on 2019 and 2022 results on state tests and the National Assessment of Educational Progress, are nuanced, especially on why students fell further behind in some districts than others.

Yes, schools that shut down or went remote for longer tended to see more of an impact. “But the losses varied widely,” writes Chalkbeat’s Kalyn Belsha in a report on the study, “and many districts that went back in person had bigger losses than districts that stayed remote.”

Advertisement

Have standardized tests lost their luster?

The story in Education Week carried a provocative headline: “State test results are in. Are they useless?”

So, are they? The story doesn’t go so far as to declare standardized tests a waste of time, but it cautions against using the latest results to evaluate schools and keep track of student performance.

It quotes testing experts and consultants to the effect that you shouldn’t compare spring 2021 test results with those from previous years. In spring 2020, Indiana and most other states canceled their tests. In spring 2021, many schools were online, and test participation was uneven.

ILEARN results show pandemic had an impact

The Indiana Department of Education released 2021 ILEARN assessment results Wednesday for students in grades 3-to-8. I don’t want to read too much into standardized test scores, especially during a pandemic, but here are some thoughts.

COVID-19 clearly impacted learning, as everyone expected it would. Between 2019 and 2021, the share of students who scored proficient on the tests declined by about 8 percentage points in English/language arts and by about 11 percentage points in math. The share of students who were proficient in both English/language arts and math declined from 37.1% to 28.6%. (The test wasn’t given in 2020).

This isn’t entirely an apples-to-apples situation, and the department cautioned against comparing 2019 and 2021 scores. For one thing, the 2019 scores included only students who were enrolled at the same school for 162 days, while the 2021 scores apparently included all students who were tested. State officials said they’re thinking of 2021 as a “new baseline” for measuring future improvement.

A-to-F school grades to continue

The Indiana State Board of Education is almost certain to continue the state’s unfortunate policy of using A-to-F grades to rate schools, judging by a framework that the board received this week.

The draft accountability framework was presented and briefly discussed at Tuesday’s board meeting. Board staff, who wrote the document, insisted it isn’t set in stone and that it will be up to the board – with input from stakeholders and the public – to decide how the system will work.

“We are trying to be transparent,” said Ron Sandlin, the board’s senior director of school performance. “The point of the framework is to spur conversations about these ideas.”

But the very first recommendation in the document is that A-to-F grades continue. The justification: “Issuing a fair and transparent summative rating ensures communities can quickly assess school performance and establishes effective incentives for schools.”

Note that the school grading system is being developed by the State Board of Education, with most of its members appointed by the governor. Superintendent of Public Instruction Jennifer McCormick, who heads the Indiana Department of Education, has favored a different approach to accountability.

The stakes are the problem

Indiana’s ILEARN scores have been made public, and the freakout is underway. I guess we should be grateful. A decade ago, business leaders and newspaper editorial writers might have pointed to the scores as evidence that schools were broken. Now the consensus seems to be that the test is broken.

Here’s another possibility. Maybe the problem isn’t with the test. Maybe the problem is what we do with it. Maybe it’s the high stakes, not the testing, that we should reject.

Results for the new ILEARN assessment were released today during a meeting of the State Board of Education. As expected, the rate at which students were found to be proficient was considerably lower than the passing rate on ISTEP, Indiana’s previous test.

ILEARN results: déjà vu all over again

Here we go again. Indiana has a new standardized test, the results sound bad, and educators are calling on the state to hold off on imposing consequences on schools or teachers using new test scores.

Today, Gov. Eric Holcomb joined the call for a “pause” in accountability based on the tests. House and Senate leaders concurred, which means it’s almost certain to happen. Results from the new assessment, called ILEARN, are scheduled to be made public at the Sept. 4 State Board of Education meeting.

Why not grade all schools on growth only?

Most Indiana schools earn A-to-F grades on a formula that gives equal weight to performance and growth on standardized tests. But schools in their first three years of operation – most of which are new charter schools and Indianapolis or Gary “innovation network” schools – can have their grades calculated on growth only, with no consideration of performance. Those schools have an advantage.

As Dylan Peers McCoy of Chalkbeat Indiana pointed out, it means you can’t use the grades to compare schools in a district like IPS. “Of the 11 out of 70 Indianapolis Public Schools campuses that received A marks from the state,” she wrote, “eight were graded based on growth alone.”

So why not grade all schools on growth only, not performance? It seems like that would make a lot of sense. In any given year, schools may not have a lot of control over where their students start out in their math and reading performance. What matters is, do schools help students grow?

ISTEP results are a non-story

It’s a lousy week to be an education reporter in Indiana. ISTEP-Plus test results were released Wednesday by the State Board of Education, so editors are assigning – and readers are expecting – the usual stories. Which schools did best? Which did worst? Which improved, and which didn’t?

Reporters who spend their work lives visiting schools and talking to educators and experts know this is the epitome of a non-news story. They know that years of experience and research tell us that affluent schools will have higher test scores than schools serving mostly poor students. But the stories have to be written.

It’s no surprise that low-poverty schools in the suburbs have the highest passing rates in the Indianapolis metropolitan area. They do every year. And it’s disturbing but not really shocking that barely 5 percent of Indianapolis Public Schools 10th-graders passed their tests. Three of their high schools were about to close; the tests had no consequences for the schools or their students.

That’s not to say test scores or meaningless, or that they should be ignored altogether.

Don’t look! It’s ISTEP time

It’s been said that Indiana’s ISTEP testing program is a train wreck. It’s also something like a car crash that you pass on the highway. You know you shouldn’t stare, but you can’t avert your eyes.

Scores from the spring 2017 tests were released Wednesday, and newspapers and digital news sites have already posted stories about how local and state schools fared. Admit it – we’re going to read them, even though we know in advance which schools will do well and which schools won’t.

I’ll leave the scorekeeping and analysis to others, but here are a few observations:

PDK/Gallup Poll: Views differ by race

People of color have a different view of their community schools than do white people. That’s an important take-away from the 2015 PDK/Gallup Poll, released Sunday.

For example, asked to rate the schools in their own community, 51 percent of poll respondents gave local schools an A or B. But only 23 percent of African-American parents and 31 percent of Hispanic students gave their local schools an A or B.

Maybe that’s to be expected: Blacks and Hispanics are more likely than whites to live in economically struggling communities with under-resourced schools. But for years, the PDK/Gallup Poll has highlighted the fact that a majority of parents think local schools deserve an A or B – the message being that most parents are satisfied with local public schools. It turns out that’s only partly true.

And African-Americans differ from whites on other topics and issues: They are:

• More likely to think test scores are an important measure of school effectiveness.
• Less sympathetic to the “opt-out” movement and less likely to exempt their own children from testing.
• More supportive of having schools teach the Common Core State Standards.

The PDK/Gallup Poll tends to produce similar headlines every year: Americans rate their local schools highly, they favor charter schools and choice but are skeptical of testing and accountability schemes, etc. But this year’s poll added a web-based component that let the pollsters break down some results by race and ethnicity and political party loyalty. That gives a better picture of the public’s attitudes.