The state testing data released Friday were supposed to permit the most sophisticated and detailed assessment ever of the Illinois public education system.
Newly revamped as a result of federal reforms, the state’s annual report card is designed to ferret out schools whose high average test scores might hide the fact that certain groups of students–such as low-income, disabled or minority children–are struggling.
It is also supposed to show whether schools tested all of their pupils–or tried to boost scores by telling low achievers to stay home on exam day.
But the data are so riddled with errors–at least 34,261 mistakes involving about 75 percent of Illinois schools–that it is virtually impossible to draw the meaningful conclusions that educators had hoped for and that the federal No Child Left Behind Act requires, according to a Tribune analysis.
State officials acknowledge they may have mistakenly placed 368 schools on a federal failure list because of data mistakes. The roster includes such perennial all-stars as Northside College Prep High School in Chicago, Butler Junior High School in Oak Brook and Carleton Washburne Elementary in Winnetka.
Most of the errors came from misidentifying students’ ethnic background, income level, special education status or English language skills. But schools also made mistakes in calculating enrollment and the percent of teachers considered qualified.
By far, the greatest number of errors came from the misidentification of low-income students. About 9 percent of the state’s 234,696 low-income test-takers were not identified as such on test booklets, making it impossible to accurately calculate the academic performance of low-income students across Illinois.
State officials decided last week to let 458 schools–about 12 percent of state schools–submit corrections to the data. But it will take until February to recalculate the statistics, and even then the final state report card will include thousands of mistakes from the remaining schools, the Tribune found.
“I think people will legitimately question the validity of the state testing system and the validity of the No Child Left Behind Law because of all these errors,” said Robert Rice, principal of North Ridge Middle School in Danville, which was erroneously tagged by the state as a failing school because of a data error. “How can we ask kids to meet high standards on one hand, and then on the other hand release a report card that is so messed up?”
State Superintendent of Education Robert Schiller acknowledged there are problems with the current report card but said it would cost the state at least $400,000 to correct every error and reissue it.
Schiller points out that much of the information is reliable, including schools’ overall pass rates in reading and math, and how well students performed at each grade level. He also contends that the picture of how well various subgroups did on the tests won’t change much once the fixes are made.
“At the end of the day, I don’t think there will be much variance,” he said.
But Lawrence Aleamoni, a statistician who advises Arizona on its student-testing program, disagrees.
“To make a claim like that … is not anchored in any data, any statistics, any probability,” said Aleamoni, a professor at the University of Arizona.
Aleamoni also warned that the mistake-ridden report cards could have significant consequences.
“Schools are being held accountable for the results, and they are taking actions because of the results, and these results may very well be flawed,” he said.
More Top Picks Best Digital Alarm Clocks With Dimmable Display
When it was signed into law last year, the No Child Left Behind Act was praised by Democrats and Republicans alike as a way to hold schools accountable for the performance of all children, not just a majority of students.
For decades, schools were judged by their overall scores, limiting failure primarily to poor and urban campuses. But the federal reform requires states to separately track minority, low-income, limited-English proficient and special education students, demanding that each subgroup within a school meet the state achievement goals.
If even one subgroup lags, the school is considered academically troubled. Continued failure can result in state sanctions. In the case of low-income schools, federal sanctions, as serious as closing, can be invoked. The law also insists that schools make sure that 95 percent of the student body and of each subgroup takes an exam.
But if students are misidentified on test booklets, it is impossible to track how subgroups are performing or if the schools met test participation standards for those groups.
Most problems in Illinois were detected while tallying the student test participation rates.
For example, the state data show that Butler Junior High School in Oak Brook tested 100 percent of its 8th graders. But the data also allege that the school tested only 93 percent of its white 8th graders–an obvious inconsistency.
Such problems can arise if a school does not ensure students are identified properly. In this case, the data indicate that only 38 of Butler’s 41 white students marked themselves as white on their test booklets. So it appeared to the state that three white children were not tested, driving down the school’s participation rate for that group.
Such mistakes are compounded when students from a particular subgroup are not identified properly, their test scores will not be counted when calculating how well that subgroup scored on the tests.
Illinois’ troubles with data collection are not unique. From Pennsylvania to Utah, state and local officials have been overwhelmed with mountains of required statistics. Local schools have submitted error-ridden data, and states have had difficulty crunching the numbers.
Kathy Christie, vice president of the Education Commission of the States, said no one knows for certain how many mistakes have been made nationwide or how many schools may have been tagged erroneously as failures. But she said schools are scrambling to fix the problems. “People read about the act, they heard about the reporting aspects of the act, but until it came back to smack them in the face, the seriousness of it didn’t sink in,” said Christie, whose group tracks whether states comply with No Child Left Behind.
Officials with the Illinois State Board of Education knew last year that they had a data collection problem, but were unsure of its breadth. A Tribune analysis of last year’s data found that 3,207 schools made nearly 58,000 reporting errors.
Schiller said he warned schools to be more vigilant this year and even offered training courses to help them understand the complicated new system. He also gave local officials 45 days in the fall to look over their data and make changes.
“No matter how much training you do, it comes down to the attention to detail and whether people are willing to put in the time to make sure the information is correct,” he said. “Obviously, many schools fell short in this regard.”
But local schools point the finger at the state, saying the data they received were not easily readable and did not show whether a school ran afoul of the law. Principals had to know the specifics of the law and make their own calculations.
Moreover, they point out that the state could have detected widespread mistakes by checking the data before sending it.
“I think the bottom line is that we can all do a better job next year,” said Hank Bangser, superintendent of New Trier High School District 203, which landed on the failure list because of a data mistake. “I know we will make a more concerted effort on our end.”
The problems with collecting and analyzing data have fueled the controversy surrounding No Child Left Behind, which has come under fire from educators who contend the law is inadequately funded and from community activists who say schools are inventing ingenious ways to skirt key provisions.
“I take it as a badge of honor that I am on the list because it shows how bogus, ridiculous, stupid, crazy this law is,” said James Lalley, principal of Northside College Prep. “How can anyone take this law seriously when it is flawed in so many ways? I think the trouble collecting this data illustrates this point perfectly.”
– – –
Errors prevalent in Illinois report card data
A Tribune analysis found almost 75 percent of public schools made mistakes when submitting test data. Most errors resulted from misidentifying students’ ethnic backgrounds, income levels and other categories.
SCHOOLS MISIDENTIFYING STUDENTS
2001-02 school year
Schools with known errors: 3,207
Schools with no known errors: 612
2002-03 school year
Schools with no known errors: 985
Schools with known errors: 2,833
BREAKDOWN OF ERRORS
Number of misclassified students
%% ERROR TYPE 2001-02 2002-03
Male 996 744
Female 1,183 910
White 9,412 4,309
Black 2,222 1,470
Hispanic 2,034 1,190
More Top Picks Best Budget Laptops For Students
Asian 610 178
Native Am. 343 0
Low-income 29,141 20,578
Disabled 5,301 2,811
Migrant 168 41
Limited-English 6,569 2,030
Total 57,979 34,261
%%
POSSIBLE CAUSE OF ERRORS
Most errors likely stemmed from incorrect reporting on forms. Some schools use preprinted labels; at others, officials mark students’ race, economic status, language proficiency and special education status.
Special-education designations
Low-income
Limited English proficiency
Migrant students
Source: Illinois State Board of Education
baiduhai