PISA testing glitch a lesson for Indiana

Back in 2006, there was an apparently minor glitch in administering the Programme for International Student Assessment, a system of student tests given by the Organisation for Economic Cooperation and Development. Some U.S. reading-test booklets were misprinted and included confusing directions.

The result: The OECD didn’t even report PISA reading scores for the United States. Officials reasoned the scores wouldn’t be valid and they shouldn’t put out bad results.

Leslie Rutkowski, an assistant professor at the Indiana University School of Education and an expert on statistical modeling, brought up the PISA experience in connection with the problems Indiana experienced with its 2013 ISTEP-Plus exam. Testing was plagued by disruptions traced to the computer servers of CTB/McGraw-Hill, Indiana’s test contractor.

“I feel like that is such a telling anecdote,” Rutkowski said. “PISA is low-stakes: The test doesn’t have consequences for students or for schools. And for something as small as a printing error to invalidate the results … that was something that really resonated with me.”

ISTEP, by contrast, is a high-stakes test – all the more reason there needs to be full confidence in its results. “We’re making decisions about whether a child graduates, whether a teacher keeps her job, whether a school stays open” on the basis of test scores, Rutkowski said. “If the data is questionable in any way, I don’t think we can use it.”

Indiana Superintendent of Public Instruction Glenda Ritz appears to agree. The state Department of Education contracted with the New Hampshire-based National Center for the Improvement of Educational Assessment to determine if ISTEP results are valid. Continue reading

Advertisements

Cheating scandals: Who will be next?

The news about the cheating scandal in Atlanta’s public schools just keeps getting worse. Teachers changed students’ answers on standardized tests from wrong to right, according to a state investigation. Students were allowed to look up answers or copy from classmates. Administrators created a “culture of fear, intimidation and retaliation” and pressured teachers to cheat.

Coming on the heels of revelations of possible cheating in Washington, D.C., and alongside concerns from Pennsylvania and other states, the news raises obvious questions: Could it happen here in Indiana? Has it happened here already?

“Because of Atlanta, the media and policymakers are going back and looking at concerns raised about their states,” Bob Schaeffer of the National Center for Fair and Open Testing told the Associated Press. “This is the top issue. When you see a story like this and see the incredible impact of the confessions, you start to look and say, ‘Hey, is there something comparable going on here?’ ”

Officials with the Indiana Department of Education are taking the idea seriously.

They recently asked CTB McGraw-Hill, the state’s primary testing contractor, to conduct an “erasure study” to look for suspicious trends of wrong answers being corrected on state tests, as happened in Atlanta, D.C. and Philadelphia.

Back in the spring, officials announced that department staff would make unannounced school visits during the March 2011 ISTEP-Plus tests, to make sure test security protocols were observed.

And a March 30 memo to the State Board of Education recounted 11 “recent violations of proper testing procedures.” Most involved teachers who devised lessons based on actual content of state exams, or who encouraged students to change answers from wrong to right.

In response, the state board gave preliminary approval in July to a rule intended to put in place stronger test-security procedures and establish procedures and penalties for security breaches. The action started a process of review and comment that will lead to adoption of the proposed rule as state law.

ISTEP-Plus is Indiana’s high-stakes test, but until now, the stakes have been high for schools but not for individuals. That’s about to change. Soon third-graders will be retained if they don’t pass the state reading test. Teachers’ salaries and job-retention prospects will depend partially on student test scores.

When there’s pressure to do whatever it takes to raise test scores, it’s not surprising that some folks will do, well, whatever it takes.

Test score success: Lessons from Lafayette?

Forget charter schools and voucher schools and waiting for Superman. The heroes of K-12 education in Indiana are the teachers – and the students! – of two public elementary schools in Lafayette.

Murdock Elementary and Thomas Miller Elementary both achieved eye-popping improvement in their students’ ISTEP-Plus scores, which were announced this week by the Department of Education.

At Murdock, 84.7 percent of students passed both the English and math sections of the test in spring 2011, up from 53.8 percent the previous year. At Thomas Miller, the pass rate increased to 85.9 percent from 61.5 percent.

And these are low-income schools, with 85-90 percent of their students qualifying for free or reduced-price lunches – the kind of student profile that often produces ISTEP passing rates of under 50 percent.

How did they do it? According to the Journal and Courier newspaper, Lafayette School Corp. officials decided two years ago to devote all their federal Title I funding to the schools with the neediest students and worst performance record, Murdock and Thomas Miller — and to supplement it with federal stimulus money.

They hired more staff and reduced class size, putting two teachers in every classroom. At Thomas Miller, the school day was lengthened by an hour. Parents had to sign school-support contracts; if they didn’t, their children were sent elsewhere. Murdock implemented a “countdown” curriculum that emphasized testing and focused attention on ISTEP. The principal and a counselor met individually with every student to update them on their progress and encourage them to do well on the state tests. Continue reading

Proposed rule would retain third-graders if they fail state reading test?

Last year the Indiana legislature considered a proposal to retain students in third grade if they failed the reading section of the ISTEP-Plus exam. But lawmakers decided not to approve the proposal, citing cost concerns.

Now the State Board of Education is about to adopt the same requirement as an administrative rule. If there are costs, the Department of Education says, they will fall on local schools, which will just have to reallocate funds in order to pay them.

The state board will conduct a public hearing on the rule at 10 a.m. Thursday (Jan. 20) at the Indiana State Library in Indianapolis. It could then adopt the rule at any future meeting. The proposed rule can be read online, as can a DOE summary and FAQ.

In addition to retention, the rule requires schools to implement reading plans that spell out goals for student achievement and interventions for students who aren’t on track. Schools will have to provide 90-minute daily uninterrupted blocks of reading instruction in the primary grades, and most will have to use a research-based core reading program certified by the state.

There are logistical issues to implementing these plans, but it’s the hammer of mandatory retention – arguably punishing kids for failing a single test – that causes concern for some educators.

“I totally agree with the goals, and that we need to have students reading by third grade,” said Cameron Rains, director of elementary instruction for the Monroe County Community School Corp. “But looking at retention and what that does, I don’t know why that is the solution you want for students.” Continue reading