An Astonishing Act of Statistical Chutzpah in the Indiana Schools’ Grade-Changing Scandal

A mathematician's guide to the news.
Aug. 2 2013 12:27 PM

The Case of the Missing Zeroes

An astonishing act of statistical chutzpah in the Indiana schools’ grade-changing scandal.

130801_DTM_IndianaGrading
Fuzzy math turns a C into an A.

Illustration by Lisa Larson-Walker

Florida Education Commissioner Tony Bennett resigned Thursday amid claims that, in his former position as superintendent of public instruction in Indiana, he manipulated the state’s system for evaluating school performance. Bennett, a Republican who created an A-to-F grading protocol for Indiana schools as a way to promote educational accountability, is accused of raising the mark for a school operated by a major GOP donor. Bennett calls that charge, which arose from emails among Bennett’s staff obtained by the AP, “malicious and unfounded” and “frankly so off base.” He offered a different explanation for why the grades for Indianapolis charter school Christel House Academy—whose mark soared from a C to an A—and 12 other schools were changed at the last minute. According to Bennett, he was just correcting a simple math mistake.

Bennett’s explanation is perfectly mathematically reasonable, and it would get him off the hook. The only problem is that the story he’s telling appears to be totally false.

Bennett told AEI’s Rick Hess, “As we were looking at the grades we were giving our schools, we realized that state law created an unfair penalty for schools that didn't have 11th and 12th grades. Statewide, there were 13 schools in question had unusual grade configurations. The data for grades 11 and 12 came in as zero. When we caught it, we fixed it.”

Advertisement

Bennett’s stated rationale makes sense. Here’s an analogy. You’re teaching a course with three exams, and each student’s overall exam grade for the class is computed as an average of her three individual exams

(1/3) * (exam 1) + (1/3) * (exam 2) + (1/3) * (exam 3)

But what happens if a student misses exam 3, with a justified absence, and the test can’t be made up? Then we have what’s called a “missing data” problem—we have to infer something about the student’s performance without the full complement of data we have for everybody else. One natural approach is to compute that student’s overall exam grade as the average of the exams she did take:

(1/2) * (exam 1) + (1/2) * (exam 2).

What you shouldn’t do is give the student a zero for the exam and average it into her grade.

Christel House had started out as a middle school and was adding one high school grade each year; for the 2011-12 school year, it served students through the 10th grade. High schools, according to Indiana statute, were to be graded on four metrics, averaged like so:

30% * (English scores) + 30% * (algebra scores) + 30% * (graduation rate) + 10% * (college and career readiness score)

But, as Bennett said in an official statement, “Christel House only served students in grades K-10, thus the graduation rate and college and career readiness measures could not be calculated because the school did not serve grades 11 and 12.” So it faced a missing data problem, like the student who had to miss an exam.

Indeed, it wouldn’t be fair to count those scores as zero! The obvious fix, just as with the student, is to grade such a school on the two scores that it does have:

50% * (English scores) + 50% * (Algebra scores)

Does that mean Bennett really did get railroaded, and he was just fixing an obvious error?

No—because the “fixed” version of the grade was what Indiana was already using before Bennett started tinkering with the gears. When you dig into the numbers, the story about the unfair zeroes looks like a complete fabrication.

Christel’s ninth- and 10th-grade students got brutally low scores on the English and math tests, with only 70 percent passing English and just a third passing math. Here’s how Jon Gubera, then the Education Department’s chief accountability officer, described Christel’s performance in one of the emails released by the AP:

“OK, here is their breakdown ...

They served grades K-10 in 2011-12 so they are a combined school but do not have any graduates. So their grade is a combination of the standard E/MS model and the HS 9&10 model which only counts ECA proficiency.

E/MS results: 3.00 on E/LA (no growth bonuses) and 4.00 on math (bottom 25% bonus) = 3.50 points (B)

HS results: 2.00 on E/LA (70% pass rate) and 0.00 on math (33% pass rate) = 1.00 points (D)

Final Combined results: E/MS 3.50 x .76 (76% of school is in grades 3.8) = 2.66 + HS 1.00 x .24 (24% of school is in high school) = .24. Thus overall grade is 2.66 + .24 = 2.9 (C).

Bottom line: their terrible 10th grade Algebra I results (33% passing) was the principal factor in earning a C grade."

Christel’s grade 3–8 scores came to a 3.50, or a B, and their high school scores, thanks to the algebra fiasco, were a 1.00, or a D. The school’s score is then an average of the grade 3–8 scores and the 9–10 scores, weighted according to the proportion of students in each group.

Where are the zeroes for the graduation rates and readiness score that got averaged into Christel’s score—the “unfair penalty” that Bennett claims he fixed? They’re not there, because that’s the part that Bennett, as far as I can tell, simply made up.

TODAY IN SLATE

Foreigners

The World’s Politest Protesters

The Occupy Central demonstrators are courteous. That’s actually what makes them so dangerous.

The Religious Right Is Not Happy With Republicans  

The XX Factor
Oct. 1 2014 4:58 PM The Religious Right Is Not Happy With Republicans  

The Feds Have Declared War on Encryption—and the New Privacy Measures From Apple and Google

You Might Not Be Crazy if You Think You Can Hear the Earth Humming

These “Dark” Lego Masterpieces Are Delightful and Evocative

Crime

Operation Backbone

How White Boy Rick, a legendary Detroit cocaine dealer, helped the FBI uncover brazen police corruption.

Politics

Talking White

Black people’s disdain for “proper English” and academic achievement is a myth.

Activists Are Trying to Save an Iranian Woman Sentenced to Death for Killing Her Alleged Rapist

Piper Kerman on Why She Dressed Like a Hitchcock Heroine for Her Prison Sentencing

  News & Politics
Politics
Oct. 1 2014 7:26 PM Talking White Black people’s disdain for “proper English” and academic achievement is a myth.
  Business
Moneybox
Oct. 1 2014 2:16 PM Wall Street Tackles Chat Services, Shies Away From Diversity Issues 
  Life
Outward
Oct. 1 2014 6:02 PM Facebook Relaxes Its “Real Name” Policy; Drag Queens Celebrate
  Double X
The XX Factor
Oct. 1 2014 5:11 PM Celebrity Feminist Identification Has Reached Peak Meaninglessness
  Slate Plus
Behind the Scenes
Oct. 1 2014 3:24 PM Revelry (and Business) at Mohonk Photos and highlights from Slate’s annual retreat.
  Arts
Brow Beat
Oct. 1 2014 6:39 PM Spoiler Special: Transparent
  Technology
Future Tense
Oct. 1 2014 6:59 PM EU’s Next Digital Commissioner Thinks Keeping Nude Celeb Photos in the Cloud Is “Stupid”
  Health & Science
Science
Oct. 1 2014 4:03 PM Does the Earth Really Have a “Hum”? Yes, but probably not the one you’re thinking.
  Sports
Sports Nut
Oct. 1 2014 5:19 PM Bunt-a-Palooza! How bad was the Kansas City Royals’ bunt-all-the-time strategy in the American League wild-card game?