An Astonishing Act of Statistical Chutzpah in the Indiana Schools’ Grade-Changing Scandal

A mathematician's guide to the news.
Aug. 2 2013 12:27 PM

The Case of the Missing Zeroes

An astonishing act of statistical chutzpah in the Indiana schools’ grade-changing scandal.

(Continued from Page 1)

Gubera’s computation wasn’t satisfactory for Bennett, because it didn’t give Christel an A, and—judging by the emails obtained by the AP—he had already told Christel administrators they were getting an A. No mere numbers were going to make a liar out of him!

What’s more, Bennett wrote in one of the emails, the low grade for Christel meant “legislative leadership as well as our critics of A-F are going to use this against us to undo our accountability metrics through legislation.” He went on: “If you can’t tell, I am more than a little miffed about this. I hope we come to the meeting today with solutions and not excuses and/or explanations for me to wiggle myself out of the repeated lies I have told over the past 6 months.”

Bennett’s staff swung loyally and swiftly into action; within a day they had found what they called a “loophole” in the state law that allowed them to change Christel’s grade. The statute clearly states that high schools without 12th-graders get a score made up half of the English score and half of the algebra score. What’s more, the law said schools that combined high school grades and lower grades should use a weighted average of the elementary/middle school measures and the four high school measures.

Advertisement

Here’s where Bennett’s team found the loophole big enough to drive a charter school through. A normal person would do exactly what Chief Accountability Officer Jon Gubera did—give Christel the weighted average of its elementary/middle school score, according to the rules for elementary/middle schools, and its high school score, according to the rules for high schools.

But Bennett had a better idea. Christel was, technically speaking, not a high school, so the statutory formula for the high school grades didn’t apply. But it also didn’t have all four high school measures, so, he argued, the rules for combined schools didn’t apply either. There were just 13 schools in the state that had both middle school and high school grades but no seniors. For these schools, Bennett reasoned, the Indiana education poobahs should have a free hand to set the grades however they pleased. You can guess what happened next: Bennett ruled that the ninth- and 10th-graders in these schools didn’t count at all. So it was that the offending algebra grades vanished in a puff of bureaucratic smoke. (Anne Hyslop at Ed Money Watch has an even more detailed accounting of the process, if you like watching sausage get made.)

This was an act of astonishing statistical chutzpah. Suppose the syllabus for my math class said that the final grade would be determined by averaging the homework grade and the exam grade, and that the exam grade was itself the average of the grades on the three tests I gave. Now imagine a student gets a B on the homework, gets a D-minus on the first two tests, and misses the third. She then comes to me and says, “Professor, your syllabus says the exam component of the grade is the average of my grade on the three tests—but I only took two tests, so that line of the syllabus doesn’t apply to my special case, and the only fair thing is to drop the entire exam component and give me a B for the course.”

I would laugh her out of the office. Or maybe suggest that she apply for a job as a state superintendent of instruction.

The saddest part is that I’m guessing Bennett sincerely felt he was doing the right thing. In his mind, he knew Christel was a great school, so if the scores said otherwise, the scores had to be wrong. In this respect, ironically, he ends up echoing his policy opponents, adopting the position that a mechanistic testing and scoring procedure can’t be allowed to override firsthand knowledge about teachers and schools.     

Saying this out loud wasn’t an option, so any test scores that seemed to indicate learning problems at Christel had to be eliminated from the spreadsheet with extreme prejudice. (I attempted to reach both Tony Bennett and Jon Gubera and have not received a reply from either man. If I do hear back, I will add an update to this story.)

Bennett told AEI’s Hess, “I'm a track and field guy. I run, I try to keep my weight down at about 190. Christel has been a track-performing school for a number of years. If I get on the scale one day, am doing everything the same, and am still wearing my same clothes and they fit, and the scale suddenly reads 215, I am going to question what's going on.”

But Christel wasn’t doing everything the same. It didn’t have a 10th grade before, and then it did, and whatever it was doing to teach those new 10th-graders math, it didn’t lead them to pass algebra.

Bennett could, and should, have faced up to that fact, given the school the C it earned, and delivered them some honest tough love: “You’re still doing well at the stuff you’ve always done well, but you obviously haven’t succeeded at the new stuff you’re trying to do. We believe you can do it, but until you do, your grade’s going to suffer.” Isn’t that what accountability in education means, if it still means anything at all?

Jordan Ellenberg is a professor of mathematics at the University of Wisconsin and the author of How Not to Be Wrong. He blogs at Quomodocumque.

  Slate Plus
Slate Picks
Nov. 21 2014 1:38 PM What Happened at Slate This Week? See if you can keep pace with the copy desk, Slate’s most comprehensive reading team.