How the NBA’s love for advanced stats has slowed our understanding of the game.
Oliver, now back out of the league and working for ESPN, says that he’s particularly frustrated by the lack of headway that’s been made on one of the first problems he posed on the Yahoo message board: What new metrics could be created to quantify individual defensive performance? The last decade has seen tremendous progress in understanding the offensive side of the floor, but defense—where players must constantly rotate and cover for each other—presents a much knottier problem. Oliver believes that technology is providing the raw data to solve it, but all those NBA stat gurus working in isolation against each other aren’t close to cracking the code.
Where is that raw data coming from? Cameras that weigh about a pound and can fit in the palm of your hand. They’re provided by STATS, the global information behemoth, as part of its SportVU program, and they currently hang in the rafters belonging to 15 different NBA franchises, six per arena. They record everything: How far and how fast a player runs during the game, how many dribbles he takes when he has the ball, where he shoots from, the arc of his shot, whom he’s passing to, whom he’s not passing to, the spots where he get his rebounds, the spots where others get his rebounds. It’s endless. For each second of game play, the SportVU cameras capture the location on the court of the ball and each player 25 times, according to Brian Kopp, a VP at STATS. “You have 1 million data records per game.”
STATS acquired SportVU in 2008 from an Israeli company that had originally designed it for soccer. This is the system’s third year in the NBA since being recalibrated for basketball. STATS charges teams from $75,000 to $100,000 per season for SportVU, and the program has grown in that time from four initial teams to now half the league. The result is one of the largest and richest data sets not just in sports, but in the world.
Kirk Goldsberry, a visiting scholar at the Harvard Center for Geographic Analysis who also uses spatial mapping to analyze the NBA for Grantland and on his blog, Court Vision, is one of the few civilians who’s been granted access to any of the SportVU data. He’s working with another Harvard professor, statistician Luke Bornn, and four Harvard and MIT Ph.D. students in a semester-long project to break some of it down. “We look at that data and we say this isn’t just good data, this is the best space-time data,” Goldsberry says. “It’s just an incredible amount of information, regardless of whether it’s about NBA or anything else … There’s very few people who have ever seen any data like this.”
If six people from Harvard and MIT have their hands full with SportVU, you can only imagine how teams in the NBA are dealing with it. STATS provides standard reports to help teams understand the information, but those only scratch at the surface of what’s possible. “I’d like to think we’re ahead,” Morey says, “but it is a whole new overwhelming amount of data. You need to take a different approach to it and I don’t think anyone has the killer app there—the thing that comes out of that data that gives someone a very significant edge.”
Many, including Oliver, believe the killer app is hiding in there somewhere. The challenge is that there’s so much information, it’s easy to get lost. “It’s like saying you’re going to Wal-Mart or Ikea to get something,” offers Tommy Sheppard, the Washington Wizards vice president of basketball administration. “You better know what you want, or you’re going to walk out with a ton of shit.” That each franchise is working alone—and against each other—compounds the problem. Goldsberry describes it as 30 “micro-CIAs,” all racing against each other to “procure actionable intelligence out of these haystacks of vast data.”
Which brings us back to that lingering question from Oliver’s first post on the Yahoo Message board: How to measure defense? Traditional measures—like blocks, steals, and rebounds—fail to account for the full context of each play, but SportVU can provide a more complete picture. “We can say, OK, when Roy Hibbert is near an offensive player, A) they don’t even tend to challenge him very much, and B) when they do, their field goal percentage is really low if he’s within three feet of the shot,” Goldsberry says. “And then you can look at somebody like David Lee—when he’s within three feet of a shot, those numbers are much higher.” The paper Goldsberry submitted (along with co-author Eric Weiss) to this year’s Sloan conference expands on that idea, using SportVU to quantify which NBA big men are best not only at defending shots close to the basket, but deterring those shots from being taken in the first place. When it comes to analyzing SportVU data, though, the authors note that their “paper’s methods only represent a small first step.”
In theory, the Sloan conference is where all these analysts now gather to learn from each other. But they’re no longer working together, as they once did on that Yahoo message board. Daryl Morey admits that, from an academic perspective, it would be fun to drop the iron curtain dividing all of the franchises so that everybody could work in unison to hash out what’s probably the greatest data challenge in the history of sports. “Maybe someday when we all get fired we could get together, but right now our jobs are to win for our teams, so we focus on that,” he says. “Our businesses aren’t for the public domain. Knowledge in general will slow down, but hopefully knowledge that gives us an edge will not.”