I will happily admit that some of those p-values are concerning. For the initiated, like a lot of the “psychological” factors that I study, they generally fell into the category of non-significant because of a high standard error, rather than low beta values. This is common when you have an effect that will influence people in a broad range of ways, but the fact that there’s enough signal poking its head out makes me think that there might be something to this.
To rule out a couple of competing hypotheses, I also looked at how many of these “meaningful” games each team had won over its last 15 games. The results were largely the same, mostly because winning a lot of meaningful games is more likely to happen when you play a lot of them. I also looked to see whether this was a matter of one team simply having a comparative advantage over the other in stress inoculation. I coded whether the batting team or the pitching team had more meaningful games down the stretch (or whether they were tied). In this case, maybe it’s the same effect if you’ve played two meaningful games to your opponent’s none vs. playing 15 against none. This didn’t come up significant for anything. It’s not whether you had more drama down the stretch than your opponent. It appears that the raw number of meaningful games (the amount of practice you already have in “playoff mode”?) drives the findings.
Finally, I limited the games under consideration to just Division Series games. This has the unfortunate side effect of throwing away about 70 percent of the sample of plate appearances, but then again, by the time you get to the LCS, everyone’s played some meaningful games. Maybe that’s muddying up the sample. The effects for “meaningful” games on the pitching team’s side for strikeouts (p = .271) and walks (p = .193) are still kind of worth mentioning. It’s hard to tell whether they went up based on a loss of statistical power or because the original findings were just a fluke.
So, the Braves and Dodgers are doomed, right?
No. Although since they play each other, I feel confident in predicting that one of them will lose. (See, the theory works!)
Everything in this study should be interpreted alongside the phrase “relative to expectations.” It looks like fighting for a playoff spot gives a pitching staff a little extra boost over what we might otherwise expect from them. It does not, however, guarantee a win in the series for that team. The team that had no drama down the stretch may have an amazing group of players, and those players are still amazing. Talent is still the main driver of results. And for what it’s worth, it should be said again that these findings aren’t as clean cut as I’d like them to be. They should also have the qualifier “I think there’s something here” tacked on. But I don’t want to completely shoo these findings away, either.
These data provide support (even if lukewarm support) for the idea that facing adversity down the stretch does make a team better in a measurable way. The data suggest that the effects can be large, but are probably highly variable across different players. That makes sense because… well, people are different. I’m at a loss to explain why only the pitchers on the team seem to reap the benefits, but that’s what the numbers tell us.
There have been a number of commentators who have noted that since the introduction of the wild card in 1995, the wild card team seems to have won an inordinate number of playoff series, especially for a team that was (by definition) not good enough to win its division and often did have to sneak into the playoffs in September. That can be explained by invoking the twin arguments of “small sample size” and “Well, but they were good enough to win 90 games during the regular season.” But maybe there’s something else going on. In the sabermetric community, we’re often too quick to dismiss that the human element can have an effect. Maybe stress inoculation really does work. If we’re going to call ourselves proper scientists, we need to be open to that possibility.
Recently, former pitcher and current broadcaster, (fantastic) author, and blogger Dirk Hayhurst wrote a piece in which he argued for the existence of momentum, based on his experiences as a player. Mr, Hayhurst has consistently been very open-minded when discussing advanced baseball analytics and is no old-school ideologue. Near the end of this piece, he wrote this line in which he nailed the problem exactly:
In an increasingly numbers-driven baseball environment, managing the human aspect of the game is often sold short. It doesn’t help that this human aspect typically gets wrapped in tired clichés by lumpen broadcasters.
The problem with a lot of the trite clichés invoking the human element isn’t that they are completely false. (Some of them are.) The problem is that they take something very complex and idiosyncratic like the human element within a multilayered situation like a baseball game and turn it into a bumper-sticker-sized aphorism, and when you just test the aphorism unto itself, of course it turns out to be false. When that broadcaster trots out a cliché in the next few weeks, maybe his sin isn’t that he’s peddling nonsense. Maybe he’s oversimplifying something that’s real—just more complex. I’d argue that’s what we have here.
Does momentum matter? Yeah, I think it does. At least that’s what the numbers tell me. The effect still has a lot of noise in it, and it is not magic pixie dust that turns quad-A players into Hall of Famers. If we had more data, maybe we could pull apart who might benefit from it and who might not. But if you peer deep into the numbers, it looks like there really is something there.