The Author of How Not to Be Wrong Explains How He Was Wrong

The power of mathematical thinking.
June 13 2014 4:41 PM

How I Was Wrong

Barry Bonds.
San Francisco Giants slugger Barry Bonds watches his record-setting 73rd home run on Oct. 7, 2001.

Photo by John G. Mabanglo/AFP/Getty Images

When you write a book called How Not to Be Wrong, you ought to expect to be fact-checked a little. And one of the virtues of the new, data-driven journalism currently in vogue is the habit of going back and checking one’s own old stuff. We’re not supposed to avert our gaze from the howlers in our old columns. We’re supposed to find the mistakes and learn from them.

In that spirit I’m going to use the last entry in this blog to look over some of my old columns from Slate, with special attention to the times I blew it.

Advertisement

But there were some mistakes, too. Here are the three biggest.

Barry Bonds isn’t going to break the home run record. Bonds had 39 home runs in the 88 games making up the first half of the 2001 season, putting him on pace for a record-breaking 72 homers for the year. But I knew the theory of regression to the mean, which reminds us that the league leader in home runs at midseason is likely to have been both good and lucky, and thus isn’t apt to maintain his league-leading pace. Historically, typical league-leaders only hit two-thirds as many home runs in the second half as they did in the first. If that trend held in 2001, Bonds would finish the season with 61 home runs.

In fact, he increased his pace, ending up with 73 home runs and the all-time season record. My reasoning wasn’t bad. It’s just that I’d neglected the possibility that there was another factor besides natural ability and luck that was working in Bonds’ favor.

Political polarization isn’t spinning out of control. Also in 2001, I wrote about Keith Poole and Howard Rosenthal’s DW-NOMINATE system for quantifying political polarization, and tracking it over the whole history of Congress. Their startling finding was that what appeared to be a sudden jerk upward in partisanship was actually a return to the normal levels of polarization that had prevailed for decades before a temporary period of abnormal bipartisanship kicked in for part of the 20th century.

But actually, the people decrying partisanship spinning out of control were right. By 2001, as I wrote, polarization had risen back to normal levels—but then it kept on rising. A Pew poll released Thursday shows the highest levels of political polarization the organization has ever recorded among American voters.

Why did I get this one wrong? I was too wedded to the idea that the past would repeat itself, that the plateau at a certain level of polarization that obtained in the first half of the 20th century would repeat itself. That degree of polarization represented a kind of equilibrium under the old conditions. But conditions change, and I didn’t stop to think about whether that might be happening.

Democrats won’t take control of the House of Representatives in 2006. I carefully computed a probability distribution based on Charlie Cook’s pre-election division of the House races into “leaning Republican,” “leaning Democratic,” and “tossup,” and came up with the answer that the Democrats had somewhere between a 15 percent and 50 percent chance of taking over the House. But Cook himself, looking at all the data he had available, and in light of his own experience, had a different take: “If nothing changes, I think the House will turn.”

It turned, and turned big, the Democrats not only eliminating the GOP majority but ending up with a 233–202 majority of their own.

Now it’s possible that some last-minute shift in fortune moved the needle in the Democrats’ direction. But it’s also possible—actually, likely—that Charlie Cook knows how to synthesize his data better than I do. Mathematical analysis of the human world can’t be carried by mathematicians working alone, if we want to do it right. It has to be a partnership between people who can wrangle the numbers and people with the deep domain knowledge the numbers alone don’t capture.

This is the final entry in the How Not to Be Wrong blog. I hope it’s given some kind of a sense of the range and force of mathematical ideas. And if you still want more? Well, I wrote a whole book about this stuff, featuring most of the examples on this blog, and a lot more besides.

Jordan Ellenberg is a professor of mathematics at the University of Wisconsin and the author of How Not to Be Wrong. He blogs at Quomodocumque.

TODAY IN SLATE

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Students Aren’t Going to College Football Games as Much Anymore

And schools are getting worried.

Global Marches Demand Action on Climate Change

Politics

Blacks Don’t Have a Corporal Punishment Problem

Americans do. But when blacks exhibit the same behaviors as others, it becomes part of a greater black pathology. 

Why a Sketch of Chelsea Manning Is Stirring Up Controversy

How Worried Should Poland, the Baltic States, and Georgia Be About a Russian Invasion?

Moneybox
Sept. 19 2014 1:11 PM Americans' Inexplicable Aversion to the 1990s
  News & Politics
Weigel
Sept. 20 2014 11:13 AM -30-
  Business
Business Insider
Sept. 20 2014 6:30 AM The Man Making Bill Gates Richer
  Life
Quora
Sept. 20 2014 7:27 AM How Do Plants Grow Aboard the International Space Station?
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Tv Club
Sept. 21 2014 1:15 PM The Slate Doctor Who Podcast: Episode 5  A spoiler-filled discussion of "Time Heist."
  Arts
Brow Beat
Sept. 21 2014 2:00 PM Colin Farrell Will Star in True Detective’s Second Season
  Technology
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Bad Astronomy
Sept. 21 2014 8:00 AM An Astronaut’s Guided Video Tour of Earth
  Sports
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.