James' greatest contribution to the baseball base was the simple but powerful idea that "a hitter's job is not to compile a high batting average. The job is to create runs." In the years since Carter began stumbling around with his punch cards, nothing so powerful—and certainly nothing so simple—has emerged from the gridiron. And for good reason: It's easy to determine the result of each event in a baseball game—double, out, strike—and to assign the individuals involved numbers based on the outcome, but football's not that cut and dried. For one thing, specialization—running backs have entirely different tasks than wide receivers—makes it more difficult to come up with a single, grand, unifying theory of football. But most significantly, though it's clear that every football player's end goal is to put points on the scoreboard—or, for a defender, to keep them off—it's not clear how to extrapolate a pancake block or an open-field tackle into a point total.
As demonstrated by Carter's 53-variable system, each discrete event in football is ridiculously more elaborate than its baseball parallel. While many baseball interactions can be modeled as either one-on-one (batter vs. pitcher) or one-on-zero (fielder vs. ball), each 11-on-11 football play can feature innumerable potential interactions, determined in part by the innumerable different formations and play calls available to both offense and defense. With all that built-in complexity, it's difficult to assign credit and blame to an individual—that is, to decide whether a running back gained five yards because of, or in spite of, his offensive line. Were the Denver Broncos geniuses for drafting Terrell Davis, Olandis Gary, and Mike Anderson in the late rounds—or were they just average backs running behind a great line?
Statistically inclined baseball fans complain that sportswriters evaluate players with teammate-dependent stats like RBIs, but football players can be evaluated only with teammate-dependent stats. No football statistic, with the possible exception of a kicker's touchback percentage, represents individual accomplishment. And these are just the biases when numbers get assigned: There are large swaths of the game—most notably offensive-line play—that have yet to be described with useful numbers.
Just as there are difficulties associated with quantifying individual ability, there are similar problems with analyzing teams. In football, the scoreboard changes strategy far more often and to a greater degree than in baseball. Whereas a baseball team tries to get hits and score runs in essentially the same fashion no matter what the score, if a football team is down by two touchdowns in the second half, they'll often abandon half their offense and rely solely on the pass.
Jim Schwartz knows from experience why most conventional stats make no sense. When he was a coach with Baltimore in 1996, he says, the Ravens were No. 2 in the NFL in passing because they were always behind and desperately heaving the ball to catch up. Schwartz, now the defensive coordinator with the Tennessee Titans, studied econometrics at Georgetown University, then earned what he calls a "Ph.D. in footballology" as an assistant coach with the Cleveland Browns under Bill Belichick. Schwartz took the Jamesian initiative to suss out whether certain statistics correlated with winning and was surprised to learn, after studying five years of data, that fumbles were evenly distributed between winning and losing teams. Belichick was incredulous when Schwartz reported his finding. "He was like, 'Good teams don't fumble.' "
When reviewing game film, Schwartz uses a simple grading system: He gives a plus (positive impact), a minus (negative impact), or a zero (no impact) to each player on each play. "You take those and then you can push them into an equation," he says. "You basically have an 11-variable equation and the result is yards gained. Over the season, over 1,000 plays, you can isolate a variable." Schwartz hopes to use his data to make personnel decisions: If a minus play by a defensive lineman costs the team on average more yards than a minus play by a linebacker, then perhaps linemen should be more of a priority in the draft or free agency.
At Football Outsiders, Aaron Schatz has his own sets of equations: His team offensive- and defensive-efficiency numbers take into account the score and time of the game and make adjustments for schedule strength. He's also imported concepts, like the notion of the replacement-level player, from advanced baseball analysis. Another innovative football stat Web site, Two Minute Warning, has done some regression analysis for Schwartz.
The data available for football geniuses to crunch, and the CPUs for crunching them, far surpass what was available to Virgil Carter. Around three-fourths of NFL teams shell out more than $10,000 per year for STATS, Inc.'s databases, which can spit out customized reports on everything from play selection in the red zone to passing yardage in blitz situations. But, as Bill James showed, the difficulty isn't in extracting and plotting the data, it's in sifting through the morass with the right question to determine which of the game's biases are rooted in empirical fact. At this point, all we really know is that good teams establish the run on offense (except the 2002 Tampa Bay Bucs), stop the run on defense (except the 1997 Denver Broncos), and, most definitely, don't fumble the ball away.