An Astonishing Act of Statistical Chutzpah in the Indiana Schools’ Grade-Changing Scandal

A mathematician's guide to the news.
Aug. 2 2013 12:27 PM

The Case of the Missing Zeroes

An astonishing act of statistical chutzpah in the Indiana schools’ grade-changing scandal.

130801_DTM_IndianaGrading
Fuzzy math turns a C into an A.

Illustration by Lisa Larson-Walker

Florida Education Commissioner Tony Bennett resigned Thursday amid claims that, in his former position as superintendent of public instruction in Indiana, he manipulated the state’s system for evaluating school performance. Bennett, a Republican who created an A-to-F grading protocol for Indiana schools as a way to promote educational accountability, is accused of raising the mark for a school operated by a major GOP donor. Bennett calls that charge, which arose from emails among Bennett’s staff obtained by the AP, “malicious and unfounded” and “frankly so off base.” He offered a different explanation for why the grades for Indianapolis charter school Christel House Academy—whose mark soared from a C to an A—and 12 other schools were changed at the last minute. According to Bennett, he was just correcting a simple math mistake.

Bennett’s explanation is perfectly mathematically reasonable, and it would get him off the hook. The only problem is that the story he’s telling appears to be totally false.

Bennett told AEI’s Rick Hess, “As we were looking at the grades we were giving our schools, we realized that state law created an unfair penalty for schools that didn't have 11th and 12th grades. Statewide, there were 13 schools in question had unusual grade configurations. The data for grades 11 and 12 came in as zero. When we caught it, we fixed it.”

Advertisement

Bennett’s stated rationale makes sense. Here’s an analogy. You’re teaching a course with three exams, and each student’s overall exam grade for the class is computed as an average of her three individual exams

(1/3) * (exam 1) + (1/3) * (exam 2) + (1/3) * (exam 3)

But what happens if a student misses exam 3, with a justified absence, and the test can’t be made up? Then we have what’s called a “missing data” problem—we have to infer something about the student’s performance without the full complement of data we have for everybody else. One natural approach is to compute that student’s overall exam grade as the average of the exams she did take:

(1/2) * (exam 1) + (1/2) * (exam 2).

What you shouldn’t do is give the student a zero for the exam and average it into her grade.

Christel House had started out as a middle school and was adding one high school grade each year; for the 2011-12 school year, it served students through the 10th grade. High schools, according to Indiana statute, were to be graded on four metrics, averaged like so:

30% * (English scores) + 30% * (algebra scores) + 30% * (graduation rate) + 10% * (college and career readiness score)

But, as Bennett said in an official statement, “Christel House only served students in grades K-10, thus the graduation rate and college and career readiness measures could not be calculated because the school did not serve grades 11 and 12.” So it faced a missing data problem, like the student who had to miss an exam.

Indeed, it wouldn’t be fair to count those scores as zero! The obvious fix, just as with the student, is to grade such a school on the two scores that it does have:

50% * (English scores) + 50% * (Algebra scores)

Does that mean Bennett really did get railroaded, and he was just fixing an obvious error?

No—because the “fixed” version of the grade was what Indiana was already using before Bennett started tinkering with the gears. When you dig into the numbers, the story about the unfair zeroes looks like a complete fabrication.

Christel’s ninth- and 10th-grade students got brutally low scores on the English and math tests, with only 70 percent passing English and just a third passing math. Here’s how Jon Gubera, then the Education Department’s chief accountability officer, described Christel’s performance in one of the emails released by the AP:

“OK, here is their breakdown ...

They served grades K-10 in 2011-12 so they are a combined school but do not have any graduates. So their grade is a combination of the standard E/MS model and the HS 9&10 model which only counts ECA proficiency.

E/MS results: 3.00 on E/LA (no growth bonuses) and 4.00 on math (bottom 25% bonus) = 3.50 points (B)

HS results: 2.00 on E/LA (70% pass rate) and 0.00 on math (33% pass rate) = 1.00 points (D)

Final Combined results: E/MS 3.50 x .76 (76% of school is in grades 3.8) = 2.66 + HS 1.00 x .24 (24% of school is in high school) = .24. Thus overall grade is 2.66 + .24 = 2.9 (C).

Bottom line: their terrible 10th grade Algebra I results (33% passing) was the principal factor in earning a C grade."

Christel’s grade 3–8 scores came to a 3.50, or a B, and their high school scores, thanks to the algebra fiasco, were a 1.00, or a D. The school’s score is then an average of the grade 3–8 scores and the 9–10 scores, weighted according to the proportion of students in each group.

Where are the zeroes for the graduation rates and readiness score that got averaged into Christel’s score—the “unfair penalty” that Bennett claims he fixed? They’re not there, because that’s the part that Bennett, as far as I can tell, simply made up.

TODAY IN SLATE

Politics

Don’t Worry, Obama Isn’t Sending U.S. Troops to Fight ISIS

But the next president might. 

The Extraordinary Amicus Brief That Attempts to Explain the Wu-Tang Clan to the Supreme Court Justices

Amazon Is Officially a Gadget Company. Here Are Its Six New Devices.

The Human Need to Find Connections in Everything

It’s the source of creativity and delusions. It can harm us more than it helps us.

How Much Should You Loathe NFL Commissioner Roger Goodell?

Here are the facts.

Altered State

The Plight of the Pre-Legalization Marijuana Offender

What should happen to weed users and dealers busted before the stuff was legal?

Surprise! The Women Hired to Fix the NFL Think the NFL Is Just Great.

You Shouldn’t Spank Anyone but Your Consensual Sex Partner

Moneybox
Sept. 17 2014 5:10 PM The Most Awkward Scenario in Which a Man Can Hold a Door for a Woman
  News & Politics
Altered State
Sept. 17 2014 11:51 PM The Plight of the Pre-Legalization Marijuana Offender What should happen to weed users and dealers busted before the stuff was legal?
  Business
Business Insider
Sept. 17 2014 1:36 PM Nate Silver Versus Princeton Professor: Who Has the Right Models?
  Life
Outward
Sept. 17 2014 6:53 PM LGBTQ Luminaries Honored With MacArthur “Genius” Fellowships
  Double X
The XX Factor
Sept. 17 2014 6:14 PM Today in Gender Gaps: Biking
  Slate Plus
Slate Fare
Sept. 17 2014 9:37 AM Is Slate Too Liberal?  A members-only open thread.
  Arts
Brow Beat
Sept. 17 2014 8:25 PM A New Song and Music Video From Angel Olsen, Indie’s Next Big Thing
  Technology
Future Tense
Sept. 17 2014 9:00 PM Amazon Is Now a Gadget Company
  Health & Science
Medical Examiner
Sept. 17 2014 11:48 PM Spanking Is Great for Sex Which is why it’s grotesque for parenting.
  Sports
Sports Nut
Sept. 17 2014 3:51 PM NFL Jerk Watch: Roger Goodell How much should you loathe the pro football commissioner?