Oh, people can come up with statistics to prove anything, Kent. 14% of people know that.
Aug 27, 2007 Colts Shmolts, Hype, Stats, Super Mario
Over the last week, on a couple different message boards as well as some random comment threads at the Dysfunctional Family Robinson, I’ve seen posts to the effect of “using stats to compare football players is stupid.”
Now, to the extent that someone would claim that Super Mario and Dwight Freeney had similar seasons last year based on the sacks each recorded, I would agree. That’s just dumb. But no one with two ounces of sense1 would make such a claim, so I highly doubt that’s what the makers of these anti-statistic statements are saying.
It seems like there are people claiming that statistical comparisons of football players are de facto misleading because… well… I’m not sure why. The usual reason tends to include something about “intangibles” or “not truly reflecting the game.”
My initial reaction is one of bemusement. If we all agree that statistics are context-dependent, that especially in football one has to look at the big picture, then a statistical comparison is as valid as any other and is one of the few means we have to remove nearly all the subjectivity. Why would this be stupid?
Looking more closely, though, I think it’s precisely this removal of subjectivity that causes people to bristle at purely numerical comparisons. When we take away the platitudes, reputations, and “intangibles,” all we are left with is performance. Suddenly, the player that is great because he’s “clutch” is only above average when you look at what he’s really doing between the hashmarks. In that situation, it’s easier for some people to disregard the message rather than altering their own beliefs. No surprise there; people are almost always loathe to admit that a personal belief is faulty.
The not-quite-ironic-but-still-kinda-funny thing about all of this is that we’ve seen it all before. Major League Baseball went through the same thing from the mid 80s to the late 90s. The old school cognoscenti derided the sabermetric crowd as eggheads who didn’t take things like clutch hitting into account. Remarkably enough, over the ensuing two decades, the vast majority of baseball fans came around. Today, nearly all are familiar with metrics like OPS and many are comfortable with VORP and ERA+. While none of these measurements are much more than 20 years old, only Luddites like Joe Morgan claim straight-faced that those metrics do not measure baseball production better than comparing Runs and Wins.
Given the improvements in understanding baseball that came from accepting the sabermetric offerings, I imagine that a similar movement will sweep the NFL. To be sure, guys like Football Outsiders have already started the ball rolling. Until new measurements become commonplace, however, we have to use the numbers we do have. While I will be the first to admit that raw numbers can be used to “prove” some things that simply aren’t true, that does not mean that all statistical comparisons are flawed from the start. To say that they are is the only “stupid” part of the whole debate.
1 Cowboy fans need not apply.
(click arrow to reply)
August 28th, 2007 at 9:53 am
I can kinda see the other side on this one — certain statistics are really, really stupid (ie completely misleading and not representative of the player’s value). I consider QB rating the most meaningless stat in any sport — the range seems completely arbitrary (just scale it 0-100 for god’s sake!) and it’s just an attempt to throw every other stat together and make a nice neat new one out of all of them.
I also think the biggest problem with individual stats in football is that the sport is far more team dependent than any other. Most other sports have statistics that allow you to gauge a player’s individual value — a baseball player’s on base percentage is independent of anything his teammates are doing (save that the pitcher may be pitching carefully due to the score or men on base). His strikeout rate is due to his skill or lack thereof with a bat, not his teammate’s performance. Virtually every statistic measured in football is dependent on the entire team, not the player saddled with the stat. David’s completion % would’ve been a lot lower had AJ not been able to catch as much as he can. A RB’s yards from scrimmage is hugely dependent on his OL, you get the idea… There’s also the simple problem of a tiny sample size — you get a lot better idea of a player’s true worth after 162 games than 16. And many priceless football players are specialists and see even less playing time to gauge stats from.
I think the most telling team stats in football are probably a team’s yards per play on offense and defense (rather than points scored or yards per game), their red zone performance, their third down performance, and their turnover ratio. For individual players I look at similar areas — how many yards per play, how do they perform in the red zone, can they convert a “big down” (third down being the most obvious), and most importantly can they hang on to the ball (or take it away from someone on D).
(click arrow to reply)
August 28th, 2007 at 10:02 am
I agree to the point that everything has to be taken in context. That’s why I am hoping that we eventually get good metrics like VORP and whatnot to compare players.
That said, when you put player A and player B side by side and one of them is markedly better in all areas (especially when you include stuff like turnover ratio, etc.), it’s preposterous to say that the lesser statistical player is still just as good because of some intangible quality.
Sure, you have to account for the O-line performance when talking about RB yards, but that’s why you have to look at the big picture. How did the team’s backup RB fare compared to the starter? Stuff like that.
I realize that it’s team dependent and I think any developments made in the field need to account for that, but I don’t think it’s impossible to compare two players numerically.
(click arrow to reply)
August 28th, 2007 at 10:15 am
Agreed that backups and replacement player type stats are critical to take into account. I don’t think football has an effective measure for the average player though, so it may be difficult to determine what is average and how much better one guy is. Position specialization makes it doubly challenging.
And you are 100% correct — it is silly to use “intangibles” as your crutch when a players is obviously inferior statistically to another. The only other excuse more frequently used is “it was their teammate’s fault your guy rocked/my guy sucked” (think Carr defenders for this).
I think a good start is to gauge a player’s worth per play run, rather than per game. Tackles or yards per carry/catch are perfect examples. A player will get an inordinately high number of tackles if his team is inordinately bad, esp. on offense. Remember Jamie Sharper leading the NFL in tackles? It was basically due to the offense going 3 and out every drive, so Sharper got more chances. A player on a team with a great defense will get more carries, so his YPG will be higher. By calculating their stats per offensive play run, one can gauge how they are performing in the chances given them compared to others in the NFL.
(click arrow to reply)
August 28th, 2007 at 10:25 am
I think “per play run” is a tremendous point. In fact, I plan on using that in the next comparison I do (though I have no idea when that would be).
(click arrow to reply)
August 28th, 2007 at 10:48 am
Aye, I think most analysts agree that the most telling statistic for a QB is “yards per attempt”. You have to put everything in context. Kitna can go out and throw for 4500 yards and 25 TDs because he plays under Martz’s system, but you have to keep in mind that he’s throwing the ball 35-40 times per game.
Another thing that goes overlooked is 3rd and long conversions. That, to me, is the most important thing for a quarterback. If you’re not moving the chains, it doesn’t matter what your completion percentage is (I’m looking at you, Carr).
(click arrow to reply)
August 28th, 2007 at 11:40 am
Nash:
I agree that YPA is the best gauge currently kept, and I don’t think it should be dropped, but I think yards per offensive play is helpful in a different context. It levels out the contributions of dual threat players (like both of Mario’s leftovers last year). It also accounts for plays across the context of a drive. Basically yards per attempt/catch/carry show what a player does when he has the ball in that situation — We can tell how Vince does when he passes by looking at his comp % or YPA, we can tell how he does when running by looking at his YPC. Yards per play shows how much a player contributes across the board when he’s on the field.