Probability, Pragmatism, and Football

I recommend Joe Nocera's recent New York Times Magazine piece on the failures of Wall Street's risk models and overall risk management. It's a worthwhile read of its own right. But I see a football angle.

I've long been a proponent of more rigorous analysis of football, and in particular its associated probabilities. But -- and this is why I am posting this article here -- football is simply so complex that there are limits to our current understanding. With baseball and the "sabermetric" movement, there are attainable gains: the game can be modeled largely as batter versus pitcher, and, within this dynamic, there is only so much change and a finite number of extra variables to control.

With football, however, you have twenty-two players, a variety of formations, receivers, combination blocks, movements, and areas to defend. The game itself follows a dizzying path to completion, with score changes, possessions, downs, field positions, yards to first down, and so on. It's not impossible to model, and there is much we can and have learned, but with football -- as with Wall Street -- what appears certain in your models and statistics may not tell the full story.

"Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital." - Aaron Levenstein

And, I am sorry to say, that when I read this article I did not think of the excellent work being done that is very useful to coaches and real practitioners, but instead of one group in particular: Football Outsiders. Now, I don't want to be too harsh. They do some great, interesting work, they have illuminated a number of subjects, and, if nothing else, the sheer breadth of what they have addressed will pave the way for more sophisticated analysis. And maybe criticism is unfair, as much of their current stuff is not focused on real football or what might bring real knowledge to the game, but instead on aiding the fantasy football player.

This is understandable, since fantasy football provides ready benchmarks and fantasy footballers are a much broader and more accessible audience than are football coaches. But if we want to be honest about what we're doing, then, to me, we ought to be honest that many of these statistics are for fans and do not actually help anyone (coaches, players, GMs) make decisions.

Football Outsiders' big stat is called DVOA, or Defense-adjusted Value Over Average. It's a nice stat, and certainly it is an improvement over just trying to compare Jimmy and Joe based on their total yards or yards per carry or whatever else. But it's a bit of a fan's statistic, and, further, there's a tendency to adjust it and use it in a way that reinforces what we already know. If DVOA doesn't tell us that Tom Brady and Peyton Manning are the best quarterbacks, then it gets revised until it does, or is rejected until a different measure tells us this unassailable truth. Same with teams: DVOA helpfully tells us that the 2007 Patriots, 1999 Rams, and 1996 Packers were all very good. And, all too often, as Brian of Advanced NFL stats has observed, Football Outsiders far too often "use[s] statistics like a drunkard uses a light post, for support rather than illumination." (Quoting Mark Twain). (One exception: Mike Tanier, though at times his analysis is a bit shaky, is clearly coming into his own and the fact that he has access to NFL film and his practice and diligence is starting to show through.)

Don't get me wrong, I adore statistics and, even more than stats, I think probabilities are the key to understanding almost everything about the world around us. I don't like dealing in absolutes. (Though some seem to vigorously disagree with me when I say that.) If I have to choose between what can only be referred to as the neaderthal view:

"We don't worry about numbers here. Statistics are for losers. I'm not a stat guy. I'm not interested in them, because you can do anything you want with numbers, you can manipulate them, and work around with them. Look at all the financial [problems] we're having in Wall Street right now. That's all those guys lying and playing with numbers. And now all of us are suffering. So I don't believe in numbers, because any crook can play with numbers....It angers me. You know? That's the whole thing, people play with numbers."

- Washington Redskins defensive coordinator Greg Blache

And what I consider the modern view:

"They say statistics are for losers, but losers are usually the ones thinking that. Statistics are great. Our whole game plan is based off statistics. Our management of the game is based off statistics. Our recruiting is based off statistics. Everything we do is analyzed. Is that the bottom line? No. You can't analyze the heart of Tim Tebow."

- Urban Meyer

Then clearly put me in the modern camp. So I really don't mean to denigrate the work of Football Outsiders or anyone else doing progressive work -- we certainly need more of it. Coach Blanche brings up the financial crisis, but it's not like -- even there, where the problems have been widespread -- that numbers and stats can just be dismissed out of hand. They are still useful, but you embrace the limitations: even if your statistics are imperfect, they are often better than nothing, and you must simply recognize and be aware of where the pratfalls are in how they might aid or hinder your decisionmaking. This was lost on the finance world recently, but it applies equally to football decisionmaking.

And, because I also consider myself something of a pragmatist, I nevertheless ask of those doing this work is the same question I would ask of any other idea (because the value of football statistics is simply another permutation of an idea about the value of statistics):

Pragmatism asks its usual question. "Grant an idea or belief to be true," it says, "what concrete difference will its being true make in anyone's actual life? How will the truth be realized? What experiences will be different from those which would obtain if the belief were false? What, in short, is the truth's cash-value in experiential terms?"

- William James, Pragmatism (1907).

So to me, there's a grand opportunity for these statisticians to change the way decisions are made and overall just improve the game -- to make a "concrete difference." Some statisticians have done this, most notably David Romer. His work is so far thorough, practical, and challenging.

I'm not a true statistician or econometrician. And, as I said above, one of the difficulties with doing real, relevant football statistics is that the game is quite complex, and one needs to understand it to model it (and those who understand usually can't model, and those who can model usually don't understand). One of my goals with this site is to try to help bridge this gap.

Nocera's piece begins with this quote from Peter Bernstein's book, Against the Gods: The Remarkable Story of Risk.

The story that I have to tell is marked all the way through by a persistent tension between those who assert that the best decisions are based on quantification and numbers, determined by the patterns of the past, and those who base their decisions on more subjective degrees of belief about the uncertain future. This is a controversy that has never been resolved.

And it will remain controversial -- in finance as well as football -- because the future will be paved by numbers and judgment, marching, somewhat awkwardly, hand in hand.