Turning Words Into Touchdowns
Does a player's speech predict how he'll perform in the NFL?
Imagine you're Carolina Panthers general manager Marty Hurney. Your team went 2-14 last year. You have the first pick in the NFL draft on Thursday. All of the pundits are predicting that you will take Cam Newton, the explosive Auburn quarterback who won the Heisman Trophy and led his team to the national championship. You absolutely need a quarterback to be the cornerstone of your franchise. Is Cam your guy?
You have his college stats. You have his highlight reel. You watched him work out at the NFL combine. You have his score on the Wonderlic—the controversial intelligence test that's been administered at the combine since the 1970s. Your coaches have interviewed him. Your scouts have timed him. You've asked around about his character, his on-field demeanor, his football smarts. What's left to consider?
Would you believe me if I told you that they should listen to his post-game interviews? Here's Newton discussing Auburn's opening-week victory over Arkansas State last year:
I'll pull out some of the scintillating highlights:
I'm just blessed to be in the situation and being able to make plays when it was time for me to make plays.
We're always happy with a win.
I'm happy with the accolade, but I feel that I didn't play the best game I could have played.
Could it really be true that these innocuous statements can help assess Cam Newton's pro potential? That's the assertion, mission, and business plan of an Ohio-based company called Achievement Metrics. It analyzes the speech of star college players, looking for traits such as "conceptual complexity," "need for power," and "deliberativeness." It compares similar players and correlates these traits with future performance. College wide receivers whose speech shows low levels of distrust, for example, have a greater probability of becoming Pro Bowlers than their less-trusting counterparts.
That may sound batty, but it's closely related to established social science. For decades, political scientists have counted the words and analyzed the grammar of political speeches in order to understand a leader's state of mind. In recent years, this process has become both automated and refined by algorithms and data crunching. Achievement Metrics has grown out of a company called Social Science Automation that does most of its work for the U.S. government. It is attempting to bring text analysis to nontraditional areas, like the CEO's corner office and, yes, the scouting of pro athletes.
The analysts at SSA don't say a lot publicly, as much of what they do is classified. You can get a flavor of their work, though, in a 2005 paper titled "The Distinctive Language of Terrorists." The paper discusses the technique of remote assessment, which involves gathering samples of speech and determining whether someone is likely to be a terrorist. The paper demonstrates the validity of remote assessment by analyzing the speech of political leaders and known terrorists and scoring them in such categories as self-confidence, task orientation, and distrust of others. Remote assessment sorted out the bad guys from the good with impressive accuracy—although, like all probabilistic models, it was not perfect. John Kerry, for example, was classified as a terrorist. (The speech that the study sampled came from the months in the 2004 campaign when he went negative.)
The speech cues that SSA considers important are not the obvious ones—"I want to kill all Americans!"—but rather instances of words like I and we, as well as the use of qualifiers—I maybe think I want to kill some Americans, perhaps. While that's a crude presentation of the work, you get the idea: Subtle, almost undetectable patterns can be analyzed to determine how likely a political figure is to engage in terrorist activity. Roger Hall, a consulting psychologist for SSA and CEO of Achievement Metrics, claims, "If you give us unidentified speech text, we can distinguish terrorists with 90 percent accuracy." Now you understand why they don't talk publicly about their work. Is the leader of that new group in Afghanistan blowing hot air or is he trouble? That's the kind of question the intelligence community is interested in.
Back to football. About five years ago, SSA analyst Steven Hofmann and a colleague started thinking about how their work could be applied to the NFL. At the time, the league was having trouble with a rash of arrests and suspensions, and coaches and scouts wanted to know who they could count on to stay on the right side of the law. Hofmann began collecting interviews with college stars—he needs only about two pages of text to do his analysis. He produced this chart:
Players whose language displays both a lack of self-confidence and a high degree of self-centeredness presented a greater risk of being arrested or suspended. For players in the upper-right quadrant, the risk was estimated at 30 percent. Again, this chart deals in probabilities. But if a team is making a $20 million investment in a player, it's useful to know if he shares common traits with players who have a tendency to get in trouble.
As with terrorists, there are no blatant words that signify whether a player is going to be a con. The analysis does not consider regionalisms or racial inflections—in fact, most of that kind of speech is corrected in the transcripts that Achievement Metrics sifts through. These are the kinds of quotes under the microscope: "I tweaked my knee in the third quarter" or "I should have cut across the field earlier." And here are some of the words that the algorithm sifted and analyzed to produce its result: nice, hard, short, cornerbacks, talent, comfortable, catch.
What this chart does ask you to do is believe that spontaneous speech reflects our character. On one level, this is common sense, since we judge people by the words they use every day. Yet Achievement Metrics hasn't stopped there. Along with using language to predict off-field outcomes, it's also trying to parse speech for clues about on-field performance. Language, then, not only reflects our character. It reflects our potential—that future backup quarterbacks talk like backups, and future starters talk like starters.