Analyst Steven Hofmann brings up the example of the 2005 draft. The two top quarterback prospects that year were Alex Smith and Aaron Rodgers. Smith played for the University of Utah and wowed the scouts with his athletic ability at the NFL combine. He also scored an amazing 40 out of 50 on the Wonderlic intelligence test. The 49ers, who saw him as a smart, gifted athlete, made him the first pick in the draft. Rodgers, who many thought would go No. 1, had the painful experience of lingering in the green room on live television until he was selected by the Packers as the 24th pick.
Hofmann, as we know, has been gathering the college press conferences of first-round draft picks and analyzing them. Here is one of the charts on quarterbacks:
The y axis is the positive power score—the belief in one's ability to influence events and outcomes: "I refuse to consider that" or "I promise that it won't happen again." The x axis shows the in-group affiliation score, terms and expressions that indicate whether the speaker associates positively with a group: "We did a great job today" or "Our linebackers were amazing." These two traits are correlated with a player's career NFL passer rating.
What would you have done if you were the 49ers general manager on draft day in 2005? The guy you like, Alex Smith, has low scores in positive power and in-group affiliation. He talks like someone who doesn't see himself as a leader. The guy you are about to pass over, Aaron Rodgers, is at the opposite end of the spectrum. He's a guy who talks like Julius Caesar. My guess: You probably would have dismissed it as academic hooey.
Achievement Metrics is trying to persuade NFL executives that it's not selling pointy-headed gibberish. It's presented its work at the premier sports stats conference at MIT and has been approached by curious teams. At the very least, this speech research seems more persuasive than the Wonderlic, which has been shown in repeated studies to have no ability to predict future performance. One reason the Wonderlic remains despite its uselessness is that we can't measure the so-called "intangibles." The private interviews the players have with teams, the two-hour psychological exam that the Giants used to administer, the personality tests—all of these are about trying to gain some purchase on what separates a Tom Brady from a Brady Quinn.
The beauty of what Achievement Metrics is trying to do is that spontaneous speech cannot be faked. (This is in contrast to the Wonderlic, which you can prep for like the SAT.) Textual analysis is not a crystal ball, of course. But look again at the chart above. There are six quarterbacks whose positive power and in-group affiliation scores were both above the median: Aaron Rodgers, Philip Rivers, Jay Cutler, Josh Freeman, Jason Campbell, and Rex Grossman. There are also six signal-callers whose positive power and in-group affiliation scores were both below the median: Alex Smith, Brady Quinn, Matt Leinart, Vince Young, Matthew Stafford, and Carson Palmer. The former group has thrown 470 career touchdown passes. The latter has thrown 290, with more than half of those belonging to Palmer. If I were an NFL general manager, I'd feel a lot more comfortable if the guy I was drafting was in that first group.
Achievement Metrics' Roger Hall and Steven Hofmann were upfront about the fact that they deal in probabilities, not certainties. "If all the correlations were perfect," Hall told me, "We would be lying." Even so, they are refining their analyses and gaining more confidence. This year, they told me, they see a similar pattern to 2005. There are five supposedly solid prospects in this year's quarterback class—Cam Newton, Blaine Gabbert, Jake Locker, Ryan Mallett, and Andy Dalton. One of these men, they told me, is in the quadrant of players who have not gone on to great success in the NFL.
Hall and Hofmann would not identify which quarterback is in the quadrant of doom. They are keeping this year's data proprietary in the hopes that NFL teams will pay them for their research. If you were the Carolina Panthers, how much would you be willing to spend to find out whether Cam Newton scores like Alex Smith or Aaron Rodgers?