Football is the most strategic of all sports. A big part of this is the unique feature that each game is 100-150 or so unique trials—the plays. This gives rise to the art and science of play-calling, “balance,” formations, sets, set-ups, counters, and whatever else that keeps us up at night thinking about this stuff. Stepping back for a second though, I wanted to simply look at the concept of balance and how we should best achieve it. Before that, however, I wanted to emphasize what I think are the most important offensive statistics.
Yards Per Carry and Average Yards Per Pass Attempt
I've long felt that the most important rushing stat, at least in terms of 1st and 2nd down performance, was yards per carry, and not total rushing yards or anything else. Quite simply, a running back who gets 120 yards on 15 carries plays in a better offense than a RB who gets 120 yards on 25 or 30 carries.
This same logic, however, applies to the passing game. More important to me than passing efficiency, or completion percentage (by itself), yards per completion, or any other statistic is Average Yards Per Pass play (including sacks). Bud Goode, legendary football statistician to the stars (Dick Vermeil, Bill Parcells, etc) has been harping on this stat for years.
The key is that it, in effect, combines completion percentage and yards per completion. The NFL QBs who have had the highest totals ever in a season are as diverse as Joe Montana (exceptionally high completion percentage) to more long-ball throwers. It penalizes the guy who inflates his completion percentage and the guy who points to his long-balls while ignoring how inefficient he is.
These two stats converge in the most important first and second down stats, which are average yards per play. The goal is to move forward to the other guy's goal-line, continually increasing your chances of scoring a TD. Further, you really want to do this on first and second down: Third down is a defense's down. The odds are in the defense's favor, and so are the strategies. Also, the teams with the best third down conversation rates are invariably the ones who have the shortest average distance to go on third down--further emphasizing that positive first and second down yards are the key.
So the goal is to find the mix of runs and passes that maximizes your teams' average gain per play.
[Note: This is not entirely true, as passes carry more risk. Turnovers make up the most important stats of all in terms of winning, and pass plays result in more turnovers than do run plays--both more fumbles and more interceptions. The answer, however, is not to ignore passing, but instead to require a "passing premium"--your passes should average more per play than your runs to counterbalance this risk.]
Offensive Identity and a Taste of Game Theory
How good you are in "absolute terms" at running or passing is a matter of talent, scheme, and reps. My argument here is that for the sake of "balance" it doesn't matter what you're better at, but, as Carolina offensive coordinator Dan Henning says, you pick a target mix and go for that, while adjusting to the defense.
This adjusting to the defense is where game theory comes in. The basic idea is that your offense and their defense have certain strengths and weaknesses, and, for the most part, everyone knows each other's weaknesses.
Imagine you are fortunate enough to have a future All-American guy at RB. He runs for a ton of yards as a junior, and now, a year later, you're ready to ride him to a state title. But everyone else knows about this guy now. They begin stacking the line. You've got this All-American at running back and you're averaging less per carry than you did three years earlier when you had three Academic All-Americans--and no football All-Americans--splitting time at RB. What's going on? What do you do?
You pass of course. You run bootlegs, you fake it to him, and you throw the ball. But how odd you say. You have the best running back you've had in 15 years, and you wind up running less? The answer is simply that everyone else knows you have this stud RB, so they commit so much effort and defensive scheme and structure to push your expected yards per rushing play down to a manageable number, your passing opportunities increase, even if you have less talent there than years past.
This same goes for great passing teams. (Think about all the spread offense teams that have used the defense's natural tendency to play pass against four wides to their running advantage.)
This little cat and mouse game is really an extension of the Nash Equilibrium from Game Theory (the subject of the movie A Beautiful Mind, about John Nash, the concept's namesake).
Application
The idea is if you are a very good passing team you pass most of the time, then you run when it is favorable and see positive results without having had to practice it too much. Same goes vice-versa--we all know how dangerous play-action passes are from heavy run teams, especially say a veer option team.
Again, I don't think yards per rush and yards per passing attempt should be exactly equal--passes are riskier than running plays. Specifically, they more often result in lost yardage (sacks) and turn the ball over more often (both fumbles and interceptions). So you should expect your yards per pass attempt to be higher than yards per rushing attempt.
To reiterate the earlier points of how this can be counter-intuitive, look at Urban Meyer at the Florida Gators (stats below). Let's say next year, with a year in his system, the passing game stays the same but the running game improves by a full yard per play. Now, what happens? Well, first Meyer will run the ball more--less risky, same reward. But then the defense will see this and begin to step up to stop the run, and drive the average yards per run back DOWN. Yet, the defense will be weaker to the pass. The result?
Counterintuitively, the passing game yards per attempt could go up and Meyer should then actually pass more. Surprised? Just think about it: If the D had to do more to stop the run, the pass gets more attractive, so Florida starts getting maybe 6.7 or 7+ per play every time they throw it, so of course they are going to throw it, even if in absolute terms it was the run game, not the pass game, that improved. Regardless, the improvement in the run game should affect the entire offense's production, which is what is important.
The lesson? If your passing game is suddenly working better, it might not be because you are suddenly Bill Walsh. It might be because you've got a stud running back everyone wants to stop.
The point of this is that you can hang your hat on one thing, but you might be leaving production on the table by not running or passing enough.
Case-Studies:
I just pulled some basic stats off of espn.com for major college teams to give some examples. I don't mean this as a criticism of these teams since my stats include some downs like 3rd down that may inflate or skew the stats, and college football includes sacks as a running play. To counteract that I added the QB's rushing numbers to the passing stats (except for Vince Young). This may be problemmatic for Chris Leak at Florida, since Urban Meyer uses a system where the QB runs the ball, but Leak was not particularly good at this and did not run near as much as Meyer's previous quarterbacks.
I fairly randomly selected these teams, though I did want to highlight teams of interest and on different ends of the spectrum.
Texas Tech
Pass-happy Mike Leach at Texas Tech attempted 697 passes for 4857 yards, averaging 6.97 yards per pass attempt. (I also recognize how many of these are shovels and the like but I'm just being simplistic.)
They ran the ball 172 times for 1040 yards, or 6.05 per rushing attempt.
So we compare 6.97 per pass to 6.05 per rush. Putting the two together the average yards per play is 6.77. We can see you can make an argument that they should have passed MORE, since that would have raised their average yards per play, but a passing premium of about a yard seems about consistent with most other teams.
The result? Tech, for all its crazy stuff, is pretty balanced.
Florida Gators
Next I looked at the Florida Gators. They got 2801 yards on 490 passing attemts (5.72 average) and 1680 yards rushing on 350 attempts (4.71). Together, the total yards per play was 5.33. Again we see about a yard of "passing premium" indicating that Urban was pretty balanced but that his team was not as productive, on a per play and total basis, as Tech.
Minnesota Gophers
Let's look at the Gophers: They ran it 586 times for 3247 yards (5.54), and threw it 347 times for 2690 yards (7.75).
That's the biggest passing premium we've seen, over 2.25 yards. Unless Minnesota is extremely risk averse, it appears that the Gophers should have passed more than they did. This result makes sense with what we said above: Minnesota had one of the best backs in the country, Maroney, and another guy who got 1000 yards. Their QB, some guy named Cupito, I didn't even remember. But defenses and defensive coordinators know the same thing. They were all geared to stop Maroney and the Gophers zone run game.
Should they have gone pass happy? No, of course not. Yet, imagine if they had thrown 30-50 more passes instead of runs (only 2-4 more per game). With more passing, the yards per pass attempt would have gone down, but I don't imagine it would have gone down to less than 6 yards like the rushing average. Also, yards per rush would have probably gone up as well. Thus, Minnesota likely would have been more productive to the point of 3 or more points in several games. In the Big 10, that is the difference between winning and losing.
The fact is that Minnesota's strength was definitely running the ball, but everyone else knew it too: Minnesota could have seen some easy success in the passing game and helped out their offense in total by throwing a bit more.
[Note: My numbers are rough so I'm not really trying to criticize Minnesota per se, just use them as an example.]
Southern Cal
These numbers are less helpful for the truly dominant teams (and less important, being smart about things matters less when you've got all the best talent). Nevertheless, let's look at the teams in the National Title Game.
USC threw 523 passes for 4193 yards = 7.88 yards per pass attempt
and 474 rushes for 3344 yards = 7.06 per rush attempt
This indicates that USC, no surprise, was very balanced and efficient in its playcalling. Maybe they should have run a bit more since that "passing premium" was kind of low, but USC is also a very efficient passing team and they do not turn the ball over very much, so they can have a smaller passing premium and get away with it.
However, the stat that jumped at me was 1740/200 = 8.7. As in 8.7 yards per rushing attempt, as in Reggie Bush's yards per rushing attempt. As in, handing the ball off to Reggie Bush had a greater expected gain than did throwing the football, which is just unheard of. This implies that USC should have handed it to him more. Now there are other issues, like durability, and Reggie's receiving prowess, but that is such a substantial number you will not see anything like that.
Texas Longhorns
Texas's stats were interesting too.
336 passes for 3083 yards = 9.18 yards per pass attempt
605 rushes for 3574 yards = 5.9 per rush attempt
That's a huge discrepency--that dwarfs Minnesota's number earlier. Texas' numbers may be skewed because it was on the good-end of a lot of blowouts and probably ran the ball much more in the second half. Nevertheless, coupled with the fact that Vince Young was the nation's passing efficiency leader, this implies that Texas probably held Vince's hand to much and should have let him throw more (or he should have stayed in the pocket and thrown more). Especially since as a runner Young averaged nearly 7 yards a carry, better than all but one of Texas' running backs. This exceptionally high passing yards per attempt number is probably correlated with Vince's running ability--the D had to take men out of position to spy him on pass plays.
Conclusion
To reiterate, my stats here are a bit on the simple side but the point is not the stats, it's the thinking: Typically a fan or coach looks at numbers like 9 yards per pass and 6 yards per rush and says "well, you don't run it as well you throw it." I think the right response, though, is "you ran it too much" or "you didn't throw the ball enough." That's a very different approach. It makes perfect sense though. It's recognizing that you're coaching against a smart person on the other side who knows where your strengths are, and then exploiting that to your advantage.
I remember someone asking Hal Mumme when he was at Kentucky about how his teams' yards per carry had dropped around a yard or so from the season before. The reporter was incredulous and turned red faced at Mumme's response: Mumme told him that he saw the same thing, and that to fix it he would throw the ball more. The reporter cut him off and essentially called him an idiot, mentioning that everyone knows you run better by simply running more (wear them down!). I'm pretty sure Mumme's point was that he coached a passing team, and if his yards per carry was going down, at least one reason was that the defense was spending too much time on the run and that he, as playcaller, was not taking advantage of passing game weaknesses defenses were leaving open.
Monday, 10 July 2006
Sunday, 9 July 2006
Run/Pass Balance - Response to Comments II
As I said in my last post, some excellent comments and I want to thank all contributors. Most of the critiques dealt with either how my analysis was too general and failed to account for down and distance, leads, and the many other football variables and second an interesting discussion regarding the "passing premium" and how to get an idea of how much riskier a pass really is than a run.
Situational Football
Many commentators pointed out that the numbers I used for the case-studies did not account for the wide variety of circumstances in a football game or they simply pointed out their situation by situation method for calling plays, such as hoping to get 4 yards on first down and then splitting the distance on second.
First, I apologize if I was unclear in my last post but I am a big believer in "thin-slicing" football data to get as specific as possible to as many different situations while keeping an eye on sample size. I simply used the broad year-long data to get a ball-park idea and to refer to teams most readers are familiar with. My hope is that any coach who applies the average per play analysis to his team will separate out the data into 1st and 10 between the 40s, red zone, 40 and in, while eliminating huge blowout games, etc.
Each coach will have to use judgment to decide what data is and is not relevant, but I never intended the only calculation to be based off year-long data. I simply assumed that using basic data would be easier for demonstration.
The Case for Average Gain Per Play
The more interesting issue though is why are we using this data at all rather than lower benchmarks like trying to just get 4 yards per play and then ignoring any gain over and above that. My response is two-fold: One, the data bears out that to win games offenses, at least in competitive games before getting a lead, should strive to score points, and second, that teams who get first downs score points, and some basic analysis helps show that teams that average more per play--even when riskier--get more first downs than teams who trade off average gain for risk reduction (i.e. too many running plays).
I'll come back to this again later, but here is a quote from Carolina Panthers Offensive Coordinator Dan Henning (one of the best offensive minds ever, he learned from greats like Sid Gillman and even tutoted Charlie Weis in football offense) that I pulled right from the Carolina Panthers playbook:
(Emphasis added) This is very insightful, and what he does not say is instructive. He does not say that you run the ball to be physical, to "establish the run" or we throw to be exciting, or for any reason x, y, or z. All those things might be true, but he simply says that we throw to score points, but it's risky. Running is less risky so we we would run the ball more than 60%, but if we did that we would not score enough points to win.
As we know, the main reason for this is that runs typically do not average as much per play as passes do (unless you've got Reggie Bush). Henning also notes that pass plays have more risk, which I will discuss later in passing premium section.
Thus, we see that the goal is to maximize average gain per play while keeping risk in mind. The risk/return tradeoff in playcalling is one I've thought about quite a bit, most notably regarding the Sharpe ratio here, here, and here.
I also posted an article based on a contribution from one of my readers, Brad Eccles. Brad, using a normal distribution, calculated the probability of getting a first down in three plays based on various average or expected gains and standard deviations (where standard deviation is a measure for a play's riskiness, i.e. passes will have gains/losses that can fluctuate from -10 to +20 or more with regularity, while runs tend to be bunched more tightly around their average gain with fewer losses and fewer big gains).
I know this diagram is a bit complicated, but the very steep slope on the right side means that average gain far dominates lower risk when it comes to getting first downs.
The basic gist of this is that in a world without turnovers you would simply try to maximize your average gain per play, and if you could always average more per play by rushing or passing, you would do that (though both rushing and passing have diminishing returns, and at some point you're going to be better off throwing a pass or running even if you hardly ever do it).
In the next section I'll talk about passing's risk per play over rushing, but this is an important conclusion. It implies that if you didn't have to worry about interceptions or turnover risk, then you would not care about run/pass balance at all and you really would just try to maximize your average gain per play whether by running or passing.
(Again, before anyone points out that the rules are different for third downs: I'm well aware, this kind of thinking is mostly for first down and 2nd and 5 and such, which make up the lion's share of all downs actually played. There are only a few third and shorts during a whole season. As I explain in my third Sharpe ratio article, you look at third and fourth downs as success/failure not as gains per play.)
Thus, with the insight that the goal on 1st and (most) 2nd downs should be maximizing average gain per play whether it is from running or passing, I'll turn to the passing premium and turnover risk, or trying to understand how much riskier a pass actually is and how this affects run/pass balance.
The Passing Premium and Turnover Risk
My original run/pass article argued that a football game is a series of Nash Equilibriums: Every down is a little contest and each side must figure out its best strategy using a mixture of runs and passes or different defenses (a "mixed strategy") and should pick the mixture of runs and passes that maximizes (or minimizes, if on defense) average gain per play, subject to the "passing premium," which requires that passes yield more than runs because of their greater risk.
But what is this risk, and how big should the premium be?
Using rough season-long numbers, we saw that Texas Tech and Florida were at about a yard, Southern Cal was a little less, NFL teams average about 2.5 yards, and Minnesota was around 2.25. I argued that Minnesota should have probably passed more, but that was somewhat speculative. To really know we need to better be able to figure out a value for this passing premium. The other problem is it is going to vary team by team, as some teams throw more interceptions than others, etc. Still, the better we understand the risk components the better shot we have.
Again, look at what Henning said:
Henning, at least implicitly, seems to argue that he'd be a lot more comfortable with throwing every down if he never got sacked and never threw interceptions. I think we can go a bit farther and, once we've isolated some of the factors, maybe go back and gather some numbers. I'll label this risk "turnover risk," though it maybe should be called "passing turnover risk minus running turnover risk" as it is the risk of throwing versus running, since the only riskless play has the QB just fall down, and even on that the snap can go wrong.
1. Interception Risk - Per play basis. This article and reader Brad Eccles note that 50 yards per interception could be a decent approximation; so you'd subtract 50 yards from your passing total every time you threw an interception.
2. Fumble Risk - I don't have great data on this, but QBs tend to fumble more than RBs (think about turnover machine Kurt Warner) and receivers are quite vulnerable as well so, a priori, I think there's a greater risk of fumbling on passing plays. [As a coaching point, I always stressed to my QBs to keep both hands on the ball when in the pocket. This makes a huge difference.]
3. Third Down Risk - I think this is the hardest one. As one reader noted, even poor rushing teams are rarely in third and 10, as failed runs usually net at least one or two yards. However, two failed passes in a row (which happens 10-15% of the time even with good passing teams) results in third and 10, which is hard to convert.
For example, if a team has a 65% completion percentage (assuming it stays the same on each down, which I know is somewhat unrealistic), then there is a 12.25% chance that the team will throw two incompletions in a row. If the team has a 55% completion percentage then there is a 20% chance they will wind up in 3rd and 10. This is a very real problem as it can significantly reduce the probability of converting for a first down, as this graph detailing third down efficiency shows:
[Hat tip: MGOBlog]
To properly calculate the risk of failed third downs from a pass attempt on first or second one would have to use the chance of an incompletion along with the reduction in the chance of converting for a first down, and then convert this to a yardage value. I have some ideas how to compute this but I'll save it for another day.
4. Sack Risk - Sack risk has been shown to be somewhat overblown, and its importance depends on down and distance and where the ball is on the field. Still, a sack on third down can (a) see a direct loss of points if it takes one out of field goal range or reduces the chance of converting a field goal, and (b) hurt overall field position, and using some of the analyses that have been done you can hang a point value on the difference between your opponent starting on his 35 and his own 45.
5. Injury Risk - I'm not sure how to quantify this, but in the NFL it might be the dominant reason why the passing premium seems to run higher than in college. Few NFL QBs make it through entire seasons without missing time, and more passes mean more hits which means more injuries. Even if it is not the hits, it could be the chance for injuries from someone rolling over the QB's leg, etc. I do think this of major concern in the NFL, while not as much in High School of College.
6. Uncertainty - I think this is an important one, at times overvalued and other times under valued. While doing some reading several commentators noted that the number of interceptions even NFL QBs threw varied wildly. Also, several teams with high and and seemingly inefficient passing premiums like Texas and Ohio St had QBs who entered the season with question marks about their passing ability. They ended up being more effective and less risky than originally thought (Vince Young lead the nation in pass efficiency) so this could partially explain the outsized passing premium. Compare Brett Favre, whom no one expected to throw 29 interceptions. Green Bay no doubt underestimated turnover risk per pass last season with a future Hall of Famer at QB.
Conclusion
I think these insights are powerful enough to where we can make most, but not all, playcalling decisions with them in mind. As we learn more we can begin approximating or building a model for turnover risk, to more accurately determine what is a sound passing premium per team. Until then, we do know that the most important factors in winning games are turnovers, third down percentage, and average gain per play (and explosive plays and red zone percentage). Taking that into account, on the majority of downs--first and most second downs--coaches should try to maximize their average output, while requiring a high enough passing premium to reward them for the increased risk from throwing the ball.
This is because the data and analysis show that it is average gain, not low standard deviations (like on run plays) that get first downs. However, turnovers and negative plays directly tie into points for the other team. Balance, then is not a matter of how many runs and how many passes, but how good you are at both and making sure you are rewarded for passing's increased risk as this is the way to more first downs, more points, and more wins.
Situational Football
Many commentators pointed out that the numbers I used for the case-studies did not account for the wide variety of circumstances in a football game or they simply pointed out their situation by situation method for calling plays, such as hoping to get 4 yards on first down and then splitting the distance on second.
First, I apologize if I was unclear in my last post but I am a big believer in "thin-slicing" football data to get as specific as possible to as many different situations while keeping an eye on sample size. I simply used the broad year-long data to get a ball-park idea and to refer to teams most readers are familiar with. My hope is that any coach who applies the average per play analysis to his team will separate out the data into 1st and 10 between the 40s, red zone, 40 and in, while eliminating huge blowout games, etc.
Each coach will have to use judgment to decide what data is and is not relevant, but I never intended the only calculation to be based off year-long data. I simply assumed that using basic data would be easier for demonstration.
The Case for Average Gain Per Play
The more interesting issue though is why are we using this data at all rather than lower benchmarks like trying to just get 4 yards per play and then ignoring any gain over and above that. My response is two-fold: One, the data bears out that to win games offenses, at least in competitive games before getting a lead, should strive to score points, and second, that teams who get first downs score points, and some basic analysis helps show that teams that average more per play--even when riskier--get more first downs than teams who trade off average gain for risk reduction (i.e. too many running plays).
I'll come back to this again later, but here is a quote from Carolina Panthers Offensive Coordinator Dan Henning (one of the best offensive minds ever, he learned from greats like Sid Gillman and even tutoted Charlie Weis in football offense) that I pulled right from the Carolina Panthers playbook:
Football, in any classification is a percentage game. A Quarterback who goes against percentages too often will fail. He'll have to be extremely lucky. No one figures to be that lucky due to so many extenuating circumstances involved in a 22 man game.
The following rules for play calling have been established for the Panthers to reduce the margin for tactical error. Errors in play calling will kill us quicker than mistakes in any other phase of football.
60% run
40% pass
The above percentage between pass and run is the healthy approach to pro football in any tightly played football game. To run more than 60% of the time will result in low scoring unless we are definitely superior. To pass more than 40% could mean costly losses as the result of failure in pass protection with loss of ball possession and field position due to interceptions.
(Emphasis added) This is very insightful, and what he does not say is instructive. He does not say that you run the ball to be physical, to "establish the run" or we throw to be exciting, or for any reason x, y, or z. All those things might be true, but he simply says that we throw to score points, but it's risky. Running is less risky so we we would run the ball more than 60%, but if we did that we would not score enough points to win.
As we know, the main reason for this is that runs typically do not average as much per play as passes do (unless you've got Reggie Bush). Henning also notes that pass plays have more risk, which I will discuss later in passing premium section.
Thus, we see that the goal is to maximize average gain per play while keeping risk in mind. The risk/return tradeoff in playcalling is one I've thought about quite a bit, most notably regarding the Sharpe ratio here, here, and here.
I also posted an article based on a contribution from one of my readers, Brad Eccles. Brad, using a normal distribution, calculated the probability of getting a first down in three plays based on various average or expected gains and standard deviations (where standard deviation is a measure for a play's riskiness, i.e. passes will have gains/losses that can fluctuate from -10 to +20 or more with regularity, while runs tend to be bunched more tightly around their average gain with fewer losses and fewer big gains).
I know this diagram is a bit complicated, but the very steep slope on the right side means that average gain far dominates lower risk when it comes to getting first downs.
The basic gist of this is that in a world without turnovers you would simply try to maximize your average gain per play, and if you could always average more per play by rushing or passing, you would do that (though both rushing and passing have diminishing returns, and at some point you're going to be better off throwing a pass or running even if you hardly ever do it).
In the next section I'll talk about passing's risk per play over rushing, but this is an important conclusion. It implies that if you didn't have to worry about interceptions or turnover risk, then you would not care about run/pass balance at all and you really would just try to maximize your average gain per play whether by running or passing.
(Again, before anyone points out that the rules are different for third downs: I'm well aware, this kind of thinking is mostly for first down and 2nd and 5 and such, which make up the lion's share of all downs actually played. There are only a few third and shorts during a whole season. As I explain in my third Sharpe ratio article, you look at third and fourth downs as success/failure not as gains per play.)
Thus, with the insight that the goal on 1st and (most) 2nd downs should be maximizing average gain per play whether it is from running or passing, I'll turn to the passing premium and turnover risk, or trying to understand how much riskier a pass actually is and how this affects run/pass balance.
The Passing Premium and Turnover Risk
The conclusion that your run/pass mix should be whatever maximizes your average gain per play assumes that runs and passes are equally risky, which of course is not true, as the old adage quoted tells us. But how much riskier are passes, exactly? And what does this say for run/pass balance?
Three things can happen when you pass, and two are bad.
My original run/pass article argued that a football game is a series of Nash Equilibriums: Every down is a little contest and each side must figure out its best strategy using a mixture of runs and passes or different defenses (a "mixed strategy") and should pick the mixture of runs and passes that maximizes (or minimizes, if on defense) average gain per play, subject to the "passing premium," which requires that passes yield more than runs because of their greater risk.
But what is this risk, and how big should the premium be?
Using rough season-long numbers, we saw that Texas Tech and Florida were at about a yard, Southern Cal was a little less, NFL teams average about 2.5 yards, and Minnesota was around 2.25. I argued that Minnesota should have probably passed more, but that was somewhat speculative. To really know we need to better be able to figure out a value for this passing premium. The other problem is it is going to vary team by team, as some teams throw more interceptions than others, etc. Still, the better we understand the risk components the better shot we have.
Again, look at what Henning said:
To run more than 60% of the time will result in low scoring unless we are definitely superior. To pass more than 40% could mean costly losses as the result of failure in pass protection with loss of ball possession and field position due to interceptions.
Henning, at least implicitly, seems to argue that he'd be a lot more comfortable with throwing every down if he never got sacked and never threw interceptions. I think we can go a bit farther and, once we've isolated some of the factors, maybe go back and gather some numbers. I'll label this risk "turnover risk," though it maybe should be called "passing turnover risk minus running turnover risk" as it is the risk of throwing versus running, since the only riskless play has the QB just fall down, and even on that the snap can go wrong.
1. Interception Risk - Per play basis. This article and reader Brad Eccles note that 50 yards per interception could be a decent approximation; so you'd subtract 50 yards from your passing total every time you threw an interception.
2. Fumble Risk - I don't have great data on this, but QBs tend to fumble more than RBs (think about turnover machine Kurt Warner) and receivers are quite vulnerable as well so, a priori, I think there's a greater risk of fumbling on passing plays. [As a coaching point, I always stressed to my QBs to keep both hands on the ball when in the pocket. This makes a huge difference.]
3. Third Down Risk - I think this is the hardest one. As one reader noted, even poor rushing teams are rarely in third and 10, as failed runs usually net at least one or two yards. However, two failed passes in a row (which happens 10-15% of the time even with good passing teams) results in third and 10, which is hard to convert.
For example, if a team has a 65% completion percentage (assuming it stays the same on each down, which I know is somewhat unrealistic), then there is a 12.25% chance that the team will throw two incompletions in a row. If the team has a 55% completion percentage then there is a 20% chance they will wind up in 3rd and 10. This is a very real problem as it can significantly reduce the probability of converting for a first down, as this graph detailing third down efficiency shows:
[Hat tip: MGOBlog]
To properly calculate the risk of failed third downs from a pass attempt on first or second one would have to use the chance of an incompletion along with the reduction in the chance of converting for a first down, and then convert this to a yardage value. I have some ideas how to compute this but I'll save it for another day.
4. Sack Risk - Sack risk has been shown to be somewhat overblown, and its importance depends on down and distance and where the ball is on the field. Still, a sack on third down can (a) see a direct loss of points if it takes one out of field goal range or reduces the chance of converting a field goal, and (b) hurt overall field position, and using some of the analyses that have been done you can hang a point value on the difference between your opponent starting on his 35 and his own 45.
5. Injury Risk - I'm not sure how to quantify this, but in the NFL it might be the dominant reason why the passing premium seems to run higher than in college. Few NFL QBs make it through entire seasons without missing time, and more passes mean more hits which means more injuries. Even if it is not the hits, it could be the chance for injuries from someone rolling over the QB's leg, etc. I do think this of major concern in the NFL, while not as much in High School of College.
6. Uncertainty - I think this is an important one, at times overvalued and other times under valued. While doing some reading several commentators noted that the number of interceptions even NFL QBs threw varied wildly. Also, several teams with high and and seemingly inefficient passing premiums like Texas and Ohio St had QBs who entered the season with question marks about their passing ability. They ended up being more effective and less risky than originally thought (Vince Young lead the nation in pass efficiency) so this could partially explain the outsized passing premium. Compare Brett Favre, whom no one expected to throw 29 interceptions. Green Bay no doubt underestimated turnover risk per pass last season with a future Hall of Famer at QB.
Conclusion
I think these insights are powerful enough to where we can make most, but not all, playcalling decisions with them in mind. As we learn more we can begin approximating or building a model for turnover risk, to more accurately determine what is a sound passing premium per team. Until then, we do know that the most important factors in winning games are turnovers, third down percentage, and average gain per play (and explosive plays and red zone percentage). Taking that into account, on the majority of downs--first and most second downs--coaches should try to maximize their average output, while requiring a high enough passing premium to reward them for the increased risk from throwing the ball.
This is because the data and analysis show that it is average gain, not low standard deviations (like on run plays) that get first downs. However, turnovers and negative plays directly tie into points for the other team. Balance, then is not a matter of how many runs and how many passes, but how good you are at both and making sure you are rewarded for passing's increased risk as this is the way to more first downs, more points, and more wins.
Run/Pass Balance - Response to Comments I
Lots of very good comments, insightful and thought-provoking. Thanks to all who responded to the blog, by email, or otherwise.
Before directly responding, I'll digress for just a moment. The interesting thing to me about football is its complexity. Baseball can essentially be modeled as a two-man game between pitcher and hitter and each's job never really changes throughout the game, it's simpler to identify the key baseball statistics that increase winning--on-base percentage, walks, and others that increase runs scored and decrease runs given up over time.
Football, on the other hand, is full of "noise." Twenty-two players on every play, different strategies, down and distance, leads, and even injury concerns all muddle the analysis. Further, there are only a handful of games each season, so we don't really get a large enough sample to know what works and what doesn't. All is not hopeless though. While baseball can maybe be thought of as a maximization problem for runs with a few variables, football is more like decision theory--many variables, changing circumstances, and lots of uncertainty.
This trait--that football decision-making is overwhelmingly complicated for what is otherwise a very simple game--leads many to throw their hands up and fall back on "common wisdoms" and truisms ("the only stat that matters are wins"). I argue, however, that the correct response is to try to be as objective as possible and get smarter about how you analyze those decisions, and then to use informed judgment. As Charlie Weis says, at some point, play-calling is more art than science, but the best artists have thought it all through.
Lastly, doing the kind of analysis I did in my last article is really just another way to attack the objective/subjective problem in football. Football games are wildly odd events; every single play in a half can be a statistical anamoly, and information is flies at you faster than you can process. Good teams adjust in-game, but the best adjusting teams have already thought of all the scenarios during the week.
In a previous post, I compared Thomas Schelling's work on decision theory to Bill Walsh and his progeny's gameplanning revolution--the real hallmark of the West Coast Offense. Walsh many times has discussed how important it is to make as many decisions as possible in the cool confines of the coaches office in order to be rational and objective rather than subjective and subject to human passions and biases.
The approach I set forth to rethinking balance and average output per play is just an attempt to do the same thing--pursue objectivity--in a different way. I think this idea is best illustrated with an example from New England Patriots coach Bill Belichick. According to several accounts, Belichick realized several rather simple facts. First, winning teams convert their third downs at a rate higher than the NFL average. Second, some teams are consistently better than others. Turning to league data, his own director of research (the fact that the Patriots have one should be a tip-off to the rest of us) and some academic work, he realized that good teams almost exclusively run in short yardage situations. In other words, on the whole, NFL playcallers--the highest paid most experienced coaches in the world--overvalued passing and undervalued running in short yardage situations. Play-callers simply didn't take into account how frequent incompletions were compared to stuffed run plays.
This result is not a total surprise, even from conservative coaches. On 3rd and 1 a QB rolls right, tosses it to the flat to the fullback . . . who drops it. On 4th and short the QB drops back, he has a guy in his face, but the Tight-end is wide open . . . and the ball sails high. Then, on 4th and 1, with the line stacked, the coach calls a dive . . . stuffed. Which one does he receive the media flak for? Criticism from other coaches? Is hardest on himself?
In the first scenario the fullback is the goat (he should have caught it!), on the second the QB (how hard is it to hit the open guy?) or the line (if the guy hadn't gotten through the QB would have thrown it fine), while on the run, it's unimaginative playcalling.
After doing his homework, however, Belichick had no choice but to conclude that all that was subjective bias: The bottom line is that runs work better in those situations, and he's run the ball to being a league leader in those situations over the last several seasons.
This is a perfect example of rethinking the entire approach to a situation in a way supported by theory and data to achieve greater success. Is it revolutionary? Is it upending all football? No of course not. But it was not a widely held belief and despite all the best and most well paid minds in the game preparing every week, he was one of the first to simply stop and say that running the ball simply works better in short yardage than does the pass. My hope is to start down the road of something like this kind of insight regarding our concept of balance on "bread and butter" downs like 1st and 10, 2nd and 5, and the like.
Before directly responding, I'll digress for just a moment. The interesting thing to me about football is its complexity. Baseball can essentially be modeled as a two-man game between pitcher and hitter and each's job never really changes throughout the game, it's simpler to identify the key baseball statistics that increase winning--on-base percentage, walks, and others that increase runs scored and decrease runs given up over time.
Football, on the other hand, is full of "noise." Twenty-two players on every play, different strategies, down and distance, leads, and even injury concerns all muddle the analysis. Further, there are only a handful of games each season, so we don't really get a large enough sample to know what works and what doesn't. All is not hopeless though. While baseball can maybe be thought of as a maximization problem for runs with a few variables, football is more like decision theory--many variables, changing circumstances, and lots of uncertainty.
This trait--that football decision-making is overwhelmingly complicated for what is otherwise a very simple game--leads many to throw their hands up and fall back on "common wisdoms" and truisms ("the only stat that matters are wins"). I argue, however, that the correct response is to try to be as objective as possible and get smarter about how you analyze those decisions, and then to use informed judgment. As Charlie Weis says, at some point, play-calling is more art than science, but the best artists have thought it all through.
Lastly, doing the kind of analysis I did in my last article is really just another way to attack the objective/subjective problem in football. Football games are wildly odd events; every single play in a half can be a statistical anamoly, and information is flies at you faster than you can process. Good teams adjust in-game, but the best adjusting teams have already thought of all the scenarios during the week.
In a previous post, I compared Thomas Schelling's work on decision theory to Bill Walsh and his progeny's gameplanning revolution--the real hallmark of the West Coast Offense. Walsh many times has discussed how important it is to make as many decisions as possible in the cool confines of the coaches office in order to be rational and objective rather than subjective and subject to human passions and biases.
The approach I set forth to rethinking balance and average output per play is just an attempt to do the same thing--pursue objectivity--in a different way. I think this idea is best illustrated with an example from New England Patriots coach Bill Belichick. According to several accounts, Belichick realized several rather simple facts. First, winning teams convert their third downs at a rate higher than the NFL average. Second, some teams are consistently better than others. Turning to league data, his own director of research (the fact that the Patriots have one should be a tip-off to the rest of us) and some academic work, he realized that good teams almost exclusively run in short yardage situations. In other words, on the whole, NFL playcallers--the highest paid most experienced coaches in the world--overvalued passing and undervalued running in short yardage situations. Play-callers simply didn't take into account how frequent incompletions were compared to stuffed run plays.
This result is not a total surprise, even from conservative coaches. On 3rd and 1 a QB rolls right, tosses it to the flat to the fullback . . . who drops it. On 4th and short the QB drops back, he has a guy in his face, but the Tight-end is wide open . . . and the ball sails high. Then, on 4th and 1, with the line stacked, the coach calls a dive . . . stuffed. Which one does he receive the media flak for? Criticism from other coaches? Is hardest on himself?
In the first scenario the fullback is the goat (he should have caught it!), on the second the QB (how hard is it to hit the open guy?) or the line (if the guy hadn't gotten through the QB would have thrown it fine), while on the run, it's unimaginative playcalling.
After doing his homework, however, Belichick had no choice but to conclude that all that was subjective bias: The bottom line is that runs work better in those situations, and he's run the ball to being a league leader in those situations over the last several seasons.
This is a perfect example of rethinking the entire approach to a situation in a way supported by theory and data to achieve greater success. Is it revolutionary? Is it upending all football? No of course not. But it was not a widely held belief and despite all the best and most well paid minds in the game preparing every week, he was one of the first to simply stop and say that running the ball simply works better in short yardage than does the pass. My hope is to start down the road of something like this kind of insight regarding our concept of balance on "bread and butter" downs like 1st and 10, 2nd and 5, and the like.
Subscribe to:
Posts (Atom)
