Is Tebow the reason the Broncos are winning? Every third news item or blog post I come across seems to be pulling its collective hair out, pondering the Broncos' 5-1 run. He's a winner! No, he's no good! He can't throw! But he runs! He doesn't turn the ball over! He doesn't take sacks! He puts his team in a hole! He's clutch! On one hand this, on the other hand that. If only their were a single number, tuned precisely to tell us how much a player contributes to his team's fortunes...That would settle the question.
Fortunately, that's what Win Probability Added (WPA) does. As a starter in 2011, Tebow has a -0.18 WPA. Suppose the football gods gave us a test with the question, On net, has Tebow helped or hurt the Broncos' chances of winning since he became the starter? Show your work. I'd write down: Hurt. -0.18 WPA. And I'd be marked correct.
We could end the discussion right there if we wanted, but that's no fun.
Tebow and the entire Bronco team had a singularly terrible game against Detroit in their only loss during Tebow's stretch as the starter. Tebow himself posted a -0.22 WPA that week. Setting that aside, his total WPA in his wins comes to +0.04 wins, mildly positive. In those same games his EPA is +8.2 points, also mildly positive.
The big reason Denver is winning is defense, RB play, and special teams. However, because of Tebow's unconventional style and scheme, some of the success of the Denver running backs could rightfully be attributed to him. Plus, he may be the perfect fit for a team built around defense. He doesn't turn the ball over or take sacks, and he can use the clock by playing keep-away with his running. But keep in mind, except for his possible effect on RB success, WPA already captures these considerations.
But even in the win yesterday, he struggled almost all afternoon. He completed 50% of his passes. He had a 36% success rate. Down by 3 late in the 4th quarter, he couldn't find the end zone, and the Broncos had to settle for a tying FG, giving the ball back to SD with 1:38 to play. In overtime, Tebow struggled again. Only on his third possession, which was possible only because SD missed a long FG attempt, was he able to move the ball. Tebow had 16 yards rushing on the dirve, but the big play was Willis McGahee 24-yd run. For the day, Tebow accumulated -0.23 WPA, which was actually slightly worse than his loss against Detroit.
The best case you can make for Tebow is that, in his wins, he's usually helped his team slightly more than he has hurt it. I'm not sure if that's a fair assessment because if you only look at any QB's winning games, you'd almost always find that he's helped more than hurt.
But don't get me wrong. I enjoy watching Tebow play because I love the unconventional offense and I love the innovation. I love that John Fox fits his scheme to his personnel and not the other way around. And in an age when every minor successful play results in some ridiculous showboating, finger-wagging dance, Tebow's attitude is refreshing.
- Home
- Unlabelled
- Tebow
Tebow
By
Brian Burke
published on 11/28/2011
Subscribe to:
Post Comments (Atom)
I also enjoy seeing the Broncos run an unconventional offense, although I'm glad I'm not a Broncos fan. But your last sentence is just flat wrong - how many other players in the NFL have a pose named after them? Maybe Tebow's overt religious displays aren't as selfish as some other player's celebrations, but they're ostentatious in their own right.
Completely agree with JDA. You really whiffed with that last sentence. The lack of effectiveness, which you have amply demonstrated with your analysis, coupled with Tebow's actions after a successful play, the medias obsession, and his zealots/fans is exactly what disgusts most people.
He does the finger wagging, but in a holier-than-thou form.
I don't think Fox is doing that good a job tailoring the scheme to Tebow.
I dunno, I have trouble blaming Tebow for someone he doesn't know inventing "Tebowing" as a meme. Unless it's Tebow fault for not *stopping* with the kneeling when that happened.
I like seeing the novel offense, but Tebow himself is a bad QB and won't ever be good. He is not the worst QB in the league or anything, but he won't ever crack the top half either. Right now he is just being held up by his defense.
As for your last sentence...I don't understand. He surely showboats as much as everyone else out there, just in his own way.
Everything about the NFL suggests the sample is too small to evaluate his overall effect on the running game particularly called option plays thus his EPA and WPA are not totally accurate.
I think you guys are out to lunch. Tebow is doing the opposite of showboating.
He is explicitly giving credit for his good fortune to 1. God (i.e. his blessings, good genes, good family, good environment, etc.) and 2. His teammates, and 3. His coaches.
I look at Denver in same fashion to those teams who used the wildcat a few years ago (as I recall, mainly Miami) except Denver is "Optioning" in almost it's offensive plays. Other teams will get better at defensing it, given time reviewing game tapes; and on the other side, Tebow will get better as a passing QB given all the help he's getting from the Denver staff. It's refreshing to see a different style of play. I like it even though I'm a San Diego fan.
The gods of football might prefer a median to net because "singularly terrible" by definition wouldn't be influencing the other games. Or, an Olympic Diving approach where you throw out games against the best and worst defenses he faced. Does anything interesting pop out if you break down those numbers by caliber of opposing defense?
Using your rankings from last week:
Detroit (1)
NY Jets (7)
Oakand (11)
Miami (22)
San Diego (26)
Kansas City (31)
If you have a second, you could cut and paste that order...then scribble in wpa and epa for each game (or direct me to where I can find them and I'll do it). That might give us a visual sense of what a game against an average defense would look like. Probably won't change any conclusions, but might set the stage for expectations against Minnesota, Chicago, New England, Buffalo, KC again the rest of the way if Tebow can stay healthy. Would be cool if there's a snow game in there somewhere...really plays into Denver's strengths (Minny's indoors, the rest outdoors)...
I think even as the option gets "figured out", it still has an inherent advantage over the wildcat in that you don't have a decoy quarterback lined up at wide receiver, and the guy getting the ball is still a threat to pass - and Tebow's been improving in the passing game. It's a built-in numbers advantage compared to a normal offense.
I'd like to read more about how Fox could do a better job at tailoring the scheme to Tebow. I know there are some spread experts out there that have opinions about this.
On second thought, rather than the wildcat example I mentioned before; I looked back to see if a team in the past had success doing what Denver has done in the past 6 games, and I found the Dilfer-"led" Ravens of 2000: a tough defensive team with just enough ball-control passing offense to make it work. The only thing is, Tebow hands off to himself rather than to some running back.
Maybe we should call this "Dilfering" in honor of the earlier example of the...dare I say it...craft?
If the league stops issuing wins to only QBs, then Tebow wouldn't be the topic. At 5-1, he's going to be the headline. Especially with his astronomical 120 QB rating in 4th Quarter and OT this year. My preference would be his team wins the SB this year, and then no one has to talk about him more.
I posted this question on the Andy Dalton article, but not surprisingly this one is getting more traffic.
I was trying to explain to my father-in-law why Tebow is not the second coming of John Elway. He kept throwing out the "He just knows how to win" line. It occurred to me that maybe in the face of being amazingly mediocre overall, Tebow is a "clutch" player in some sense. That is, could we look at the highest EPA/WPA plays in a game and spot plays that could be attributed to a particular player performing well above their own average in a key situation?
In addition to Dalton's big play that was detailed in the other post, Tebow had a +.19WPA play in the third when he completed the 36 yard pass on 3-11 from the DEN 25.
I'm not saying I believe in "clutch" play in general (I don't want to be pilloried on this site), but maybe some players are just good enough when they really need to be.
ok, so it's not Tebow would seem to be the hypothesis.
Then what happened to the broncos to change them from a terrible Suck for Luck candidate, to making a run at the division title?
On a second point, tebow does a lot of stuff that is fantastic: his fake handoffs are very good and usually cause the run defenders to move the wrong way. He has actually made the option effective in the NFL. He is almost unsackable (a combination of his ability, and the defensive game plans he faced). A lot of his playmaking ability does not get into the WPA/EPA statistics (for instance, his option tosses).
I don't think "Hurt, -.18", is quite enough to mark it correct. To determine whether he's helped or hurt the team as a whole, I think you have to look a whether he's helped the team relative to the alternatives they had. The Broncos weren't starting "league average QB" before that, or could have signed him instead of starting Tebow. They were starting Kyle Orton.
And that's where it gets a bit interesting. In terms of WPA/game, Tebow has actually been better than Orton was. Both have been net negative, but Orton was more negative. However, Orton was completely average in EPA/play, while Tebow has been quite bad.
So, from a outcome-based, retrospective view, Tebow has been better for the Broncos than Orton was. However, Orton was better per play, and we could reasonably suppose that he would have done better than Tebow if he had started the games that Tebow did.
Funny how irrational and polarized opinion is about Tebow. He is a 24-year old, second-year quarterback, with a somewhat but not terribly novel skill set. In 2010, he was one of the most valuable per play (0.21 EPA/P) quarterbacks in the NFL, albeit in a very small sample. This season, in a system purportedly built around him but that has not brought out his best, he has struggled (-0.05 EPA/P) but his team has won. Could he succeed? Yes. Has he succeeded so far? Depends on the criteria. Is there much else that can be said that doesn't resort to estimations of the man, or critiques of how he experiences his faith? Not really.
I hope he can become a successful quarterback, because failure is boring.
Lest we forget, Vince Young started out just like Tebow -- as the "winner" who "just won" and turned the team into a winner by himself (as pointed out in analysis Brian linked to earlier).
Is Tebow the reason the Broncos are winning? Every third news item or blog post ...
I'm in NYC -- about as far from the center of Tebowmania as possible -- and just saw a local newspaper with with a big picture of Tebow and the caption "Teammates embrace Tebow after he led the offense to win the game over the Chargers with a last-minute scoring drive in overtime".
They scored 16 points in 73 minutes and the offense won the game.
Tebowmania is the sum of two pathologies: (1) The QB is responsible for everything, "carries the team", "wins the championship", and (2) What happens at the end of the game is somehow much more important than what happened earlier. Adam Vinatieri won the 2004 Super Bowl with yet another great clutch FG at the last moment -- after missing earlier from 31 and 36. Of course, if he'd hit the earlier easy ones... And if Tebow had played earlier with even modestly positive WPA/EPA then the game wouldn't have gone to OT for "him to win it" there.
I tried to point out the absurdity of (1) in a discussion at FOers recently to people who were seriously arguing that Petyon Manning is personally worth 10 wins a season, all by himself, citing numbers from a Colts 13-3 season as proof.
Well, say team performance is produced by 22 starting players, plus a dozen top situation players, with each starter other than the QB getting a share of performance and each of the other dozen a half share. Arithmetic tells us that if the QB produces 10 of 13 wins, 77% of positive results, then his contribution is *91 times* that of the average other starting player. Hey, Peyton is *way* underpaid! (And same for Brady, Rodgers, Brees).
The Peyton worshipers replied, "Yes, true, a QB like Peyton is literally invaluable", worth so much it can't be calculated, mere computed performance numbers like DVOA, EPA, can't measure it. Which is why the don't. That's what they actually said.
Irrational QB hero worship. Same with Tebow.
"Then what happened to the broncos to change them from a terrible Suck for Luck candidate"
Looks like the same thing that happened when the Titans got Vince Young. There are 52 other players on the roster to contribute to change, plus a big coaching staff.
And then there is all that pure randomness in game results that no fan ever wants to consider.
Couldnt disagree with this article more. If you actually believe he played worse yesterday than he did against the Lions... your on crack. If your 'formula' backs you up or comes to that conclusion... its also on crack.
BTW, the logic used to prove by DVOA that Peyton is worth 10 wins all by himself was...
~~~~
The Colts went from 3-13 before Peyton to 13-3 the year after he arrived. DVOA shows no improvement in the Colts' numbers anywhere near enough to account for 10 wins. Thus all the wins were due to Peyton. QED.
True, the Colts' passing numbers by DVOA after Peyton arrived didn't improve by anywhere near enough to account for 10 wins. Thus Peyton's invaluable ability to win 10 games cannot be captured by DVOA performance numbers. QED#2.
Ergo, great QBs win games by being great QBs (via clutchiness, inspiring teammates, and such) which is captured by no statistic other than "wins", the undeniable one. And thus their teams winning proves their greatness.
~~~~
Is the logic with Tebow any different?
(Footnote: Pythagorean says those Colts improved from 6-10 to 10-6 strength, by four wins over two years, with the arrival of a new coaching staff, Peyton, James, etc., the development of Harrison, etc...)
Responding to the last comment - WPA doesn't measure how well a player plays. It measures how much they impact the game. Tebow did play badly against the Lions in the second half (he didn't play too badly in the first half), but in the second half, the game was out of reach. So any second-half bad plays were going to score as > -0.05 WPA when they might have scored much worse had the score been closer. Similarly, when the score is close, Tebow is going to get dinged hard when he scrambles for 4 instead of 6 yards on a 3rd-and-six late in the game (as happened on Sunday), when other seemingly minor plays like good punts and 5-yard penalties are also going to to have a huge impact on the game. I mean, late in the game, you could half Tebow rushing for two yards, and then a defensive holding penalty with a first down, and then Tebow rushing for another two yards, and he'd get creamed in WPA.
So, comparing San Diego to Detroit - you can argue that Tebow had more of a (negative) impact in the San Diego game. Tebow was directly involved in a lot of plays that made victory less likely, like failed third-down conversions. And yes, it's true that many of these plays had subtleties that WPA can't measure (e.g. receiver drops), and that it's not counteracted by his positive impact on the running game, but those negative WPA plays did happen.
EPA is more about who played better, and there his Detroit score is much worse than his San Diego score. What's funny is his San Diego EPA is much closer to 0 than his WPA, and that you can argue that his San Diego performance was anti-clutch. Throughout the game he avoided mistakes, but on average, he did a little worse in the situations when it truly mattered. Yes, he had a couple of big plays at the end of regulation and in overtime, but that's in line more with what clutch *actually* is - quarterbacks don't get better in the final minutes, it's more just that that is when the rest of us are paying more attention.
Using the link BB provided in the article (didn't see the link at first because I was so focused on the article...old school style...
Detroit (1)...-0.22 WPA, -26.9 EPA
NY Jets (7)...+0.29 WPA, -0.1 EPA
Oakand (11)...-0.03 WPA, +4.6 EPA
Miami (22)...-0.03 WPA, -0.2 EPA
San Diego (26)...-0.23 WPA, -0.2 EPA
Kansas City (31)...+0.04 WPA, +1.8 EPA
Throw out the best and worst opposing defenses, and you get +0.0 in WPA and +4.1 in EPA if I'm doing this right (though negative numbers in three of four games)...outclassed by highest echelon...not well suited to exploit lowest echelon...hanging around equality otherwise.
Does that paint a more accurate picture for the gods than -.18 did, or not?
Might be better to focus in the middle of Tebow's sampling in general because they won't put the ball in his hands to do big things if things are going well (they'll just run out the clock), while he could have disasters playing from way behind. Skews full season volume more negative than a subset of "typical" games would suggest.
Where your statistics fail here you said it yourself. Your statistics give him no credit for making the other running backs better. Compare the running game between the Oakland games and you will have a sense of what he is doing. Defenses cannot focus their run defense on the rb's alone. Until your statistics take this into account they are not telling the entire Tebow story.
I think Tebow does deserve some of the credit for the running game, if nothing else because he has two key abilities that make the option-read plays possible.
1. Reading the DE - it helps the runningback by taking a defender out of the play. This causes a numbers advantage because Tebow effectively 'blocks' the DE (or at least doesn't cause a number disadvantage like most plays).
2. Legitimate running threat - every time Tebow hands off in the option-read it's only because the DE is worried about an outside run. If Tebow couldn't run the ball effectively (picture Brady running) the DE would play normally and go for the RB, knowing he or someone else could easily recover and stop Tebow.
Without both of these attributes the read-option doesn't work, so Tebow should share some of the credit on these plays. Of course, I wouldn't expect EPA/WPA to pick this up because quantifying these effects would be extremely difficult and we don't make similar adjustments on other plays, so it's better to keep them consistent. (For instance, it's highly likely Rodgers and his receivers help the running game via defensive personnel and formations in trying to stop the passing game).
However, that does mean EPA/WPA aren't the be-all, end-all to the argument, just the beginning (albeit a very excellent beginning). Personally, I think that's what makes this kind of informed debate fun, and if we keep working at it maybe someday someone will figure out how to quantify it.
All that said, it's pretty clear the defense has been the primary factor in the wins.
I would just like to say that I agree pretty much exactly with what Jim Glass said.
Tebow is Vince Young 2.0. The *maybe* 25th best QB in the league who would be looked at as a career backup/position transfer project if he played on a team with a slightly worse defense. He is around Tavaris Jackson level? If he played on a team with a significantly worse defense all we would be hearing about is how he was a huge bust and is terrible at football.
He took an bad offense where marginal starter Orton was putting up 21pts/game, and put up EVEN FEWER! (19.3). Meanwhile the defense has gone from giving up 28pts/game to 20.
To put it another way if there was an NFL redraft right now of all the rostered players I am not sure Tebow would go in the top 32 QBs. Maybe he would, but I wouldn't be shocked to see QBs like Locker, Gabbert, Mallet, Jackson, and even Orton! going ahead of him.
Denver only played him to show everyone he couldn't be a long-term solution. Unfortunately for them the defense has made that plan blow up in their face. I would be absolutely shocked if Tebow is a starting QB in 2 years.
Frank Day-
he absolutely makes the running game better. He also makes the passing game one of the worst in the league if not the worst.
Tebow gets half the credit for finding a way to win, and the other half for inspiring his teamates to win. Who can deny that the defense and special teams have stepped up their game since Tebow entered the fray? They know their QB won't give up, and they're playing with the same desire now.
It's also worth noting that the read option has been especially fruitful for McGahee. He averages over 7 yds. per carry with it, and less than 3 without it. If Denver gets a good TE in the offseason, and another decent WR, they could be unstoppable next year.
5-1 with the "worst" passing game in the league. Most Broncos fans will take that trade-off gladly.
Come on, what really makes a bad passing game. Interceptions or low yardage or losses?
Frank Day-
If your response to criticism of Tebow's long term viability is "5-1" I don't think this is really the site for you.
Here are some links that are more to your tastes:
http://www.nfl.com/standings
http://community.foxsports.com/go/forum/view/67121/114121/denver_broncos
It is far-fetched and really disappointing to see fans criticize Tebow for his attitude. That perception comes solely from what's been written about him and NOT what he does. Look at what every person who knows him has said: he's humble, he genuinely cares about others, and he wants to win.
I know it annoys many people that he truly believes his "religion", but that doesn't mean he's not humble.
I think a good perspective is this: is Tebow the right man for the job in Denver? It sure seems like it, with their solid running game and incredible defense. Is he the right man for the job next season? That depends on if they can find someone better.
Tebow is obviously not an "elite" quarterback and if you think he is a huge difference maker, then you're not watching the games. However, he really isn't an "awful" quarterback either. is he the 20th best QB? Is he the 30th? Whatever the case, he's good enough to win games if Denver keeps games low scoring.
Great article -- and comments!
Brian,
In this post it seems like you may be veering into the classic error of mistaking the map for the territory. WPA is certainly a statistically useful concept for understanding the players and the game of football, but it does contain assumptions that would prevent me from saying that it can tell us definitively in one number what Tim Tebow (or any other player for that matter) contributes to his team. I think that the assumptions that underlie WPA are particularly vulnerable in edge cases, in this case an offense that depends on either running or passing to an extreme extent.
My understanding of WPA is that it is based on an average of the results of similar situations in past games. As far as I know it doesn't attempt to differentiate between teams with different tendencies. I'm not sure is there has been an attempt to quantify this in the past. I do recall a post about teams running more when winning and passing while behind, but it seems possible to me that teams with different run/pass tendencies would have a different WP in the same situation.
My gut feeling would be that a run heavy team like the current Broncos would tend to be at an advantage compared to the "average" team in games that were closer than one score in the 4th quarter, while they would be at a big disadvantage if they were losing by more than one score.
To sum up, I think that WPA provides a very useful way of thinking about the game in aggregate, but when applied to specific situations it might be useful to keep in mind that there are factors that might cause it to be less accurate. This tends to be inherent in modeling anything, not just football.
Joshua, my original comment pointed out the weakness or the statistics in evaluating Tebow. Tebow looks awful when held to the statistical standard of the prototype NFL quarterback. But, he is showing that it is possible to win doing something other than what many consider to be "ideal".
The question here is "what are these statistics not telling us?" Here is a point of view on this that you might consider.
http://www.uh.edu/engines/epi1840.htm
Here is another statistical view that is worth considering in this discussion:
http://sportsillustrated.cnn.com/2011/writers/kerry_byrne/11/29/tim.tebow/index.html
Frank, I don't understand the desire to do gymnastics to explain why Tebow's individual numbers (be they things like NY/A, EPA/P, WPA/G or even the dreaded DVOA) don't agree with the number of wins the Broncos have since he began starting. A number of other young or old quarterbacks or middle aged quarterbacks have also won a lot more games than their individual statistics and performance suggests, such as Mark Sanchez, Joe Flacco, Young Tom Brady, Middle Age Alex Smith and Young Vince Young. In those cases everybody pretty much assumed that it was the that was explaining most of the winning. I don't see why we need to go through special gymnastics to explain Tebow's wins. Sometimes a below average quarterback will win a large number of football games!
Also, if we want to attribute credit for the running game to Tebow, surely he needs to be debited when the running game performs poorly. McGahee had very good games against Oakland and San Diego (.76 WPA and about 9 EPA, along with good success rates) and bad games against the Jets and Dolphins (-1.02 WPA and -13.5 EPA). Ball was the primary rusher vs. the Chiefs and he had a terrible game (low WPA, but 28% SR and -5.7 EPA). So I think the jury is very much out on the Broncos rushing attack being highly successful owing to Tebow. The tailbacks have had 2 good games and 3 bad ones with Tebow starting (Tebow's rushing stats are incorporated into his overall EPA/WPA/SR stats).
it was the *defense* that was explaining most of the winning. Also luck.
Christopher, it seems to me what defines a "below average" quarterback is number of wins. If statistics claim a winning quarterback to be "below average" perhaps it is the statistics that are lacking. That is the point.
Maybe 2 years from now Tebow will be sitting on the bench because teams have figured him out. But, if that is not the case, and Tebow keeps winning, then maybe we need to come up with another metric to measure QB effectiveness.
And, of course, the defense is part of the equation. Football is a team sport. Without a reasonable defense even Rogers can't win regularly. But, without Tebow, that defense, who did pretty much the same job as they are doing now at the beginning of the season didn't win much with Orton at the helm. Their one really bad game was against Detroit, a game "Tebow" lost.
A number of other young or old quarterbacks or middle aged quarterbacks have also won a lot more games than their individual statistics and performance suggests, such as Mark Sanchez, Joe Flacco, Young Tom Brady, Middle Age Alex Smith...
That is: teams win games, not QBs, as these and countless other examples show.
Sometimes a below average quarterback will win a large number of football games!
You mean his team will win a lot of games in spite of him, carrying him.
As a Jets fan I can tell you that Mark Sanchez, circa 27th-best QB in the league the last two seasons no matter whose numbers one measures by, did *not* win all those games needed to go to the Conference Championship those two years -- the Jets premier defense and top running game and special teams did and carried him there.
~~
Christopher, it seems to me what defines a "below average" quarterback is number of wins.
I'd think that defines a below average team. How does a sport in which there are 22 starters, 30+ regular contributors, and 53 on a roster, become a contest between *one man teams*? See my comment above about Peyton and some Colts fans.
Is the definition of a "below average small forward" in basketball also the number of team wins? If not, why not? He's one of only 5, not 22! If an NFL QB is worth *four* other starters -- a lot! -- the small foward still has a larger impact on the game outcome.
Why do so many fans think of pro football as a singles tennis match between QBs?
And, of course, the defense is part of the equation. Football is a team sport.
Ah, yes, a valuable afterthought that should be an up-front thought all the time. Remembering that the QB is only one of 53, many mysteries evaporate. (Even if many heros and derided failures are diminished.)
Jim when I was saying the below average quarterbacks "won" games I was saying it in the sense that every player on the team wins the games. Like Deion Branch won a Super Bowl and Greg Jennings has won 10 games this year. You know what I'm saying.
Also, passer rating is a horrible stat. Does anybody believe a touchdown is worth 90 yards and an interception over 100 yards? Why would Cold Hard Football Facts come up with a stat that was substantially similar to a stat that is really, really, really horrible.
Christopher: You might as well ask, "Why did ESPN come up with Total QBR?, a stat that is substantially similar to a stat that is really, really, really horrible."
CHFF is a narrative-based sports site with a thin veneer of statistics on top of it.
My Tebow prediction is his settle down as a RB/QB in some team's system.
I like to think of Adjusted Net Yards per Attempt as a common-sense version of passer rating.
Sacks count as a negative pass attempt
TD = +20 yards
INT = -45 yards
A better measure of how good a QB is is what a team does with him and without him. Now, that is hard to know exactly in any particular game but one can make guesses. Why people are so "up" on Tebow is in the comparison to the team's performance when Orton (the "best chance" for winning according to the coaches preseason) and Tebow (who has "turned this team around on two occasions, at the end of last season and, again, now).
This is not evidence that Tebow is a "better" quarterback than Rogers or anyone else but only evidence that this team performs at a much higher level when Tebow is playing than Orton. Would he do the same on another team? Who knows? But, he sure has, seemingly, had a huge impact on the Broncos and it is difficult to explain using the standard statistics.
Anyhow, it remains to be seen how far they will go this year or into the future with him.
Here's a take on Tebow probably nobody will like, but I think it's the truth.
Small wins, one score games, have very little predictive value for future W-L, while "big wins" (and big losses) by two scores or more have a lot of predictive value (as I documented last year on the community site).
Small wins are chance, luck. Brian Burke has shown that more than 40% of NFL game outcomes are determined by chance. These are them. The W-L record in one-score games of Vince Lombardi's Packers was exactly 50%, and of Bill Walsh's 49ers was 41%. Random chance.
Thus, small wins are *not* a sign of team strength, and a streak of them does *not* mean the team is getting better (or worse). These games don't show one team is better than the other. *Big* wins show that. There's reams of data (from all sports) showing teams that win (or lose) a lot of close games aren't as good (bad) as their WL record, and regress to their true strength -- which is their record in big decision games.
In short, to estimate the strength of a team from its W-L record, treat big decisions as Ws and Ls and one-score games as *ties*. This works well in predicting future game outcomes, and thus is an indicator of real team strength -- a better one than total W-L record including those close decisions.
As it happens I track games this way, and looking at the Broncs with Orton starting they played a hard schedule, against teams with a 58% win percentage, scoring 40% (four "tie value" games and a decisive loss to the Packers) for an Orton-led big win record of 48%.
With Tebow at QB the Broncs have one 2-score decision up and one down with four "ties" (two OT games) against 46% level opposition -- they have played 46% ball. No meaningful difference. Not in anything that predicts success going forward, i.e. team strength. That's why the "Tebow effect on winning" isn't seen in any measured stat like EPA, WPA, DVOA, completion percentage, or anything else. There isn't one.
With Tebow at QB the team has won one game on the merits, lost one on the merits, and won four coin flips. Hardly a staggering improbability. (It wasn't Tebow who won the Miami OT game when the Miami QB fumbled the ball in Denver FG range!)
This is remarkably consistent with the QB being only one of 22 starters, 30+ regulars and a 53-player roster ... explaining how just one player can have such a tremendous effect on game outcomes produced by *all of them* while it remains invisible to statistical analysis. He doesn't.
For all those who certainly still believe that 'no, no, close-game victories show winning ability, character and leadership' I'll just note that while Lombardi's and Walsh's teams were a combined sub-.500 in one-score games, the coach with the all-time *best* record in them (20 or more) was Vince Tobin ... though he went only 13-38, 25%, all his other games.
Tebow: a great deal of to-do about not much, so far. Except I really do enjoy seeing the option in the NFL. :-)
I think there's a flaw with small-win big-win in that 17-10 is different than 37-30.
I do believe that there is no such thing as clutch, and we've already seen a regression to the mean for Tebow in the last game - his EPA was better than his WPA.
There are a lot of things that can be disguised as clutch, though. Overly conservative players and coaches that don't let it rip until they really have to, punting on 4th and 1 instead of going for the points because you believe in your defense and field position, running the ball instead of throwing to the end zone in order to burn time, etc.
I don't get how all these stats account for the fact that Tebow has a MUCH better record than Orton over the past two years; and if we remove last year as being incomparable, then Tebow has a much better record THIS year with the same defense. I'm sorry, I just can't buy the argument that Tebow's starting gig just happened to coincide with the magical morphing of the Denver defense. I know WPA is supposed to account for every impact a QB can have on the game but the Denver running game has taken off with Tebow at the helm and kept the defense fresh. Tebow is not a magic man nor even an average NFL throwing QB, however, why can't analysts just admit that since there hasn't been a QB like Tebow in the era of advanced stats that there's no way to adequately measure him? I would expect stat gurus to understand that you can't use the same measuring stick on a previously never seen player in an offense that hasn't been run in 30+ years.
Hey anon. how about reading Jim Glass's excellent comment that essentially looks at a high level win loss record and determines that essentially Orton was unlucky while Tebow was lucky, when it came to whether or not the team won a close game. That covers the defense and running game. Fact is the Denver Broncos haven't shown evidence of being appreciably better or worse, therefore any argument that he improves the whole team needs further proof. The Broncos as a team aren't any better with Tebow @ QB, they just happen to be winning more. (Nice comment Jim)
RBs have gone from 4.31 YPC to 4.66 since Tebow's arrival.
Jim Glass wins.
Tebow= beware of making narratives fit small sample sizes.
Team Giveaways the last four games... corresponding to the Broncos reducing the passing load from Tebow to 22-8-21-18 after the first two starts of 27-39...
1: Denver
2: Green Bay
3: Miami, New Orleans, Dallas
4: Atlanta, Houston, San Francisco, Jax
5: Baltimore, Cleveland, New England
6: Seattle, Cincy, Pittsburgh
7: Tennessee, KC, Chicago, Oakland, Minnesota
8: Philadelphia, St. Louis, NYG, San Diego
9: Buffalo, NYJ, Wash, Carolina, Indy
10: Arizona
11: Tampa Bay
12: Detroit
People can rationalize all they want about how good these statistics really are and the only thing standing out to explain Tebow is luck (I guess the Broncos did win the luck bowl, although it didn't come about through "suck for luck").
Regardless, the fact remains that these stats do not take into account the effect of the QB on the running game nor the defense. I guess when all of the QB's are the same and their only real and believable options are to 1. hand off to someone else. 2. fake hand off to someone else and throw. 3. throw. then the effect is going to be small, the only difference might be how good a fake the QB can perform.
But, when the QB also has a real option to run himself not only can that improve the running game by itself (assuming the QB is effective), it can also improve the other runners by how it dilutes the defense (depending upon how effective the QB is at running).
So, it is easy (and reasonable) to ignore this effect when evaluating QB effectiveness looking at the prototypical QB but it is not when looking at a QB who also runs effectively - and it is not necessary to run much to have this effect if the defense believes you can and will run.
Anyhow, it seems strange to me to be arguing that the normal stats are "just fine" in evaluating the effectiveness of Tebow as a QB. (Yet, Newton, who is a similar threat, isn't seeing the same result - what sets them apart, is it really luck or something else?) At some point, if he keeps on winning regularly, when will it be necessary for the statisticians to say, "the chances of my throwing 80 heads out of a hundred cannot be attributed to 'luck'", We need to come up with a better metric.
Hmmm, Newton isn't seeing the same results? The Panthers have gone from dead last in the NFL in passing efficiency to 9th, 13th in the NFL in rush yards per attempt to 2nd, and dead last in the NFL in points scored to 17th. Unfortunately they are giving up more points than all but one other team. I'd say Newton (and some other signings like Shockey, Olson, and an apparently rejuvenated DeAngelo Williams) have had a dramatic effect on the Panthers' play.
Yes, Christopher, Newton has had a dramatic effect on the panthers play using the usual metrics. Unfortunately, it hasn't had the same effect on the number of wins that Tebow's play seems to have had on the Broncos. Is there a measurable reason to account for this difference that could be put into a statistical analysis?
Any statistical analysis - in favor or against Tebow - is almost entirely irrelevant with such a small sample. Small sample analysis always creates a statistical artifact: the small sample is prone to higher variability. This combined with our need to interpret the analysis creates a dangerous combination, one in which we are for more certain than we should be, one in which we draw far too many conclusions than we normally can. Anyone preaching more than an educated guess as to how Tebow will fair as a QB moving forward is far too confident in their conclusion.
Quote: He is explicitly giving credit for his good fortune to 1. God
So if someone has bad fortune (bad environment, bad family), is that because God doesn't like them or wants them to fail?
The idea that good things happen to a person because of God has a corollary, and both sides of the equation are reprehensible.
Anyone who believes that he has found the One True Religion, and has a personal relationship with the wisest, most powerful intellect that could ever be, is nothing like humble--it's about as arrogant an attitude as I can imagine.
Last I checked, disavowing any and all credit to oneself and instead giving it to teammates, family, and/or God is pretty selfless and humble.
He doesn't push his beliefs on to anyone...where's the harm?
feralboy12: That's one of the best comments I have ever read, on any blog/forum.
Brian, can you create a graph showing the relationship between the number of comments a post receives and whether or not the title of the post has "Tebow" in it?
feralboy12: That is one of the worst comments I have ever read, on any medium.
wow, Tebow is certainly worth his weight in media/fan interest. incredible how much interest he has sparked.
intangibles: leadership, setting an example for others to follow, facing adversity, believing in those around you, being a good sport, superior focus, work ethic, leveraging the workout, following a plan...equals winning..these are some leading indicators. Your after the fact analysis is a negative test for truth...these intangibles are a prerequesite for winning and can't be measured.
a prerequesite for winning and can't be measured.
So very intangible and unmeasurable as to perhaps exist entirely only in one's imagination, as a story one likes to tell oneself.
Got any tangible evidence otherwise? :-)
Newton has had a dramatic effect on the panthers play using the usual metrics. Unfortunately, it hasn't had the same effect on the number of wins that Tebow's play seems to have had on the Broncos. Is there a measurable reason to account for this difference that could be put into a statistical analysis?
Ah, well, Newton is only one player on a 53-player team -- some of the others have an effect on game outcomes too. E.g, they've very measurably combined to give up more points than any other team in the league except Indy. So that would be a "yes".
But there's been any impact on the overall level of the play of the Broncos, since Tebow started? I doubt it, as per my last comment above. By all evidence their quality of play has been darn near exactly as before. If two OT games go the other way -- two coin flip outcomes reverse, say the Mia QB doesn't fumble the ball in OT in FG range for Denver -- then the Broncs are 3-3 with Tebow against sub-.500 opposition, .500 being their real strength of their 53-player roster, just like before, and there is no story, no list of mysterious but invaluable "invisible intangibles" bringing victory after victory. Nothing. No sign of God intervening. This whole story comes down to two coin-flip outcomes.
But *let's say*, for argument's sake that the Broncs' level of play has seriously improved in the last six games to pile up victories -- without it being visible in Tebow's personal non-game winning quality numbers.
Why would anyone assume Tebow is the cause?
Say the defense has gotten better to win these low-scoring games -- wouldn't it be *much* more likely that the DC or Fox is the cause?
This is just like the Colts fans mentioned above who were seriously arguing that when the team went from 3-13 before Manning to 13-3 in his second year, Peyton was worth all those 10 games *personally* -- because the stats didn't show anything supporting a full 10-game rise, *including* no such improvement in passing stats. So what else could be the cause but Peyton?
Total lack of evidence supporting the premise that the QB produced the wins *proved* the QB produced the wins! Because ... what else could produce wins but the QB?
Hey, could all those invaluable intangibles producing these wins be provided to the team by, say, the new Head Coach and his new staff (in both the Indy and Denver cases?) After all, isn't it the new HC's *job* to instill character and intangibles in the team (see: Vince Lombardi). And aren't he and his staff in a much better position to do it than a new untested kid QB?
Or could the improvement in the play of the team result from, be "inspired" by any of the other 52 players on the team? No ... apparently because of the irrefutable reason that coaches and other players aren't QBs.
This the logic of rooster worship. The rooster crows, then the sun rises, so the rooster controls the sun, because ... what else is possible? Who else could fill the sun with all the intangibles it needs to get up there so early each and every morning but the rooster? He crowed and it rose, QED. Rooster worship and QB worship, conjoined twins. :-)
"feralboy12: That is one of the worst comments I have ever read, on any medium. "
He made his point in a straightforward and rather polite way. Since I am sick of ignorant bigots like yourself I won't be polite. If you honestly believe the above you:
A) Have spent almost no time on the internet or are lying.
B) Are a complete idiot and bigot.
C) Don't actually understand the way the world works.
It is a perfectly sensible and straightforward comment. That you don't like what it entails just shows the poverty and lack of quality in your belief system.
At a religious debate I once had a girl who had "miraculously" survived a kidney problem that killed 99 in 100 people tell me it was because of her family's faith and goodness. She also claimed her survival proved god existed. Apparently god doesn't care for starving Ethiopian infants, but he does care for some ill suburbanite? When I asked her whether she thought maybe some of the corresponding 99 people who did die of that same ailment (after all that it what it means for it to be a 1/100 shot) might also have been deeply religious and good with caring families, she physically attacked me and called me a heretic.
Just pure ignorance. Why not just start sacrificing the children of our enemies to Ah Kahtenal on top of a pyramid to help make the corn grow? It makes as much sense as mumbling to the invisible-sky-father.
It is Santa Claus for adults, period. If you really want to embrace that, well what are you doing on a website that is concerned with attempting to understand the truth of things? You clearly are not interested in the truth.
The Broncs and Tebow in a nutshell:
The Broncos in their first five games get thumped by the Packers like everyone else has, and otherwise play four very close games (2-pt, 3-pt, 3-pt, 5-pt) in which they go 1-3.
In their next six they win one two-score game (Oakland) get killed in one (Detroit) and play four very close games in which the teams differ by a grand total of 10 points in regulation, two are OT. In these close games they go 4-0.
Anybody who is familiar with these things knows close games like these are determined by chance -- *not* by clutch play, character, intangibles. (Unless you want to say Vince Lombardi's teams with their .500 record in one-score games, and Bill Walsh's with their .410 record, lacked character and intangibles -- while Vince Tobin's were loaded with them!)
So it comes down to this: the Broncs have played 8 games determined by chance, in the first four they were a little unlucky to go 1-3, while during the second four they've been lucky enough to go 4-0, for 5-3 total, one off .500.
That is it, the entire story. No mysterious personal intangible factors that suddenly lift the team to victory-after-victory while remaining inscrutably unmeasurable are necessary.
Chance rules the universe -- and one-score games in the NFL.
(What's that? Vince Lombardi & Bart Starr and Bill Walsh & Joe Montana couldn't reliably win close games ... but Tebow can, because he has "intangibles" they all lacked. OK, sure.)
Wow, really? My position is that people (including athletes) should have the freedom to believe what they want to believe, so long as it doesn't cause harm to others. It's rather mind-boggling that you've twisted Tebow's (public) faith into a post that states "Just pure ignorance. Why not just start sacrificing the children of our enemies to Ah Kahtenal on top of a pyramid to help make the corn grow? It makes as much sense as mumbling to the invisible-sky-father." Of course you don't mean this in reference to Tebow, but that's what we're talking about, isn't it? Is this really the time for a rant on the evil, ignorance, or plain stupidity of religion?
I'm as much of a non-believer as you are (just way less arrogant). But that doesn't give me the right to spew hatred towards those that do believe. I'm sorry some girl physically attacked you and called you a heretic...small sample size though, no? Maybe not all religious folk are like that...
I'm a big believer in freedom of expression. I don't like to tell people what they can and can't do (again, so long as they're not hurting anyone). And if you really want to argue that giving credit to teammates and God is selfish...go for it.
to win, you need to score at least one more point than the other team. tebow seems to have a knack for doing this than most QBs. i don't think other stats matter that much.
b.t.w. this also applies to a whole team. with tebow playing, broncos, as a team, figure out a way to score more than the other team... so they win.
@ 2nd to last Anon I wouldn't go so far as to slam everybody who believes in God, but Tebow's father's organization (with which Tim did his missions to the Philipines and presumably still associates) is demonstrably anti-Catholic and believes in Young Earth Creationism. The media's unwillingness to discuss either of those things and instead just sort of lump it all together as like "oh he's a man of God" is pretty maddening.
Jim, from the article: "But keep in mind, except for his possible effect on RB success, WPA already captures these considerations."
The article admits that the most obvious area in which the Broncos have changed post-Tebow is in the running game (both the QB himself and the RB's). It just seems a little strange to be arguing that it can be demonstrated that he doesn't have an impact when you admit that you aren't measuring all the areas in which he might have an impact.
Anyhow, I suspect this debate will continue for awhile. At some point one side or the other will start to have better "evidence" to support their side. Until then it will be interesting and controversial.
There are many ways of looking at this issue, and it might be helpful to remember what Denver had devolved to at the time the decision to bench Orton was made. Even when that happened, people in the field were characterizing Denver's start differently. I remember KG using the word "dismal" in an article here, at the time where BB's rankings only had the Broncos around 19th (a few spots below average). So, even HERE at Advanced NFL Stats, the guy with the Magna Cum Laude from Northwestern wasn't in line with the guy with the degree in Aerospace Engineering. Denver was either dismal or very slightly below average depending on the keys you were looking at.
Just before the benching, Denver fans watched this sequence:
*Barely getting past Cincinnati while getting outgained 382-318, and losing yards-per-play 6.1 to 5.2. The defense allowed a rookie quarterback to pass for over 300 yards without a pick.
*Barely lost at Tennessee, but got squashed in the stats...losing total yardage 333-231 and yards-per-play 5.5 to 3.7. The defense allowed the opposing quarterback to throw for 295 yards without a pick on 75% completions. Seeing this game with your own eyes probably created a different perception than just scoreboard watching.
*Got beaten badly by Green Bay 49-23. Yardage wasn't so bad because Green Bay plays some rope-a-dope on pass defense with leads. Denver lost turnovers 4-2 (3 picks), and was down 8-4 in that category within this three game set (11-5 for the year by the way at this time, meaning very sloppy football).
*Trailed San Diego at the half 23-10, with a yardage deficit that's kind of hard to believe. The Drive Chart at ESPN shows an estimate of 359-99 (Denver's TD came on a pick six of Rivers, note that drive yardage isn't quite the same as total yardage but it's a good enough ballpark).
Denver fans were watching the team collapse before their very eyes. The problems foreshadowed by the poor numbers vs. Cincinnati and Tennessee had now blown up...and the team was playing like they had thrown in the towel. 359-99 in a half! Someone seeing this could easily think "dismal" as KG did.
Now...move forward to the Oakland game. The loss to Detroit put a stop to the Tebow worship for the time being...because he clearly looked overmatched by a top quality defense. He passed the ball 39 times for only 117 yards. Coach Fox decided to move to a very run heavy approach to see if the Broncos could at least be competitive with something creative.
Since then, we have four games with:
*0-0-1-0 in turnovers (and a 1-5 edge)
*A composite edge in yards-per-play
*A 55-35 edge from scoring on drives of 60 yards or more (an edge in driving the field)
*A more aggressive pass defense (4 picks in 4 games after just 3 in the first 7), where nobody's had anything like 300 passing yards with no picks.
If you compare the last 3.5 games with Orton, to the 4 games with Tebow after the switch in styles, you get a better sense of why there's excitement about Tebow at the moment.
A clean team playing good defense isn't getting squashed in the stats. During the darkest days of Orton, a sloppy team had been getting outplayed, and had apparently thrown in the towel. This is a meaningful difference, even if the explanation doesn't fit in a nutshell.
Where you draw the lines and what stats you look at will strongly influence perception of what happened (as well as if you were watching the games with your own eyes or just reading stat summaries).
Jeff-
What does your second to last paragraph have to do with Tebow? It was a sloppy TEAM, now it is playing good DEFENSE. Neither of those things has much to do with Tebow.
Brian,
There are two things undeniably missing from your models that would clearly help put all this in context.
1. Confidence intervals
2. God
If there were confidence intervals, we could see the effects of small sample sizes and values that are not significantly different, statistics-wise.
If there was a God independent variable in your regression equation, we would see how Tebow has been able to win all those close games.
Tebow's Real WPA = Tebow's Model WPA + (x)God
I thank the Lord for giving me the opportunity and strength to write this post. And for the record, God is on board with this. I know it.
It was a sloppy TEAM, now it is playing good DEFENSE. Neither of those things has much to do with Tebow.
Hey, a Martian visiting Earth and observing the sport might think that an entire team getting less sloppy as the season progresses is the result of the new Head Coach and his staff having an increasing effect on it as they do more work with players ... with an improving defense attributable to the DC and defensive side of the new coaching staff.
But a Martian could never understand how EVERYTHING is Tebow. (Well, everything good.) Everything -- both real and imagined.
Jim Glass. I simply don't see where anyone in this discussion has attributed "everything" good to Tebow. This started based on an article that supposedly used a statistical analysis that supposedly attributed NONE of the improvement to Tebow. You have come up with this analysis in order to evaluate individuals, not teams. My part in this debate is simply to comment on the obvious weakness of this particular statistical analysis as it relates to this particular player. In fact, this weakness was admitted in the original article and then ignored.
Joshua,
The quarterback touches the ball on almost every play, so...clearly, a change from sloppines to sharpness could be attributed to the QB...particularly in the case here of a team choosing to use a lower-risk run-based approach. Denver was 3-2-2-4 in giveaways with Orton in his four full games, and he had 1 additional turnover in the first half of the game he got benched. Denver is 0-0-1-0 the last four games since switching to a run-heavy attack. Don't think it's out of line to give Tebow some credit for that if he's making good reads on the option and helping his team avoid high impact mistakes.
With defense, it's always dicey to try and draw conclusions because the synergy between offense and defense can be hard to pin down. There's certainly a case to be made for offensive ball control helping a defense stay fresh...for offenses who avoid turnovers not putting a defense behind the eight-ball in terms of field position...and just for the the fact that some teams play very soft defense when they stop caring.
Denver's defense looked like they had thrown in the towel during those six quarters when they allowed about 870 yards to Green Bay and San Diego. If you go back to the play-by-play data in the SD boxscore, you can see that defensive intensity picked up right after the change was made from Orton to Tebow. Denver sacked Rivers four times in the second half and forced a fumble that helped trim the lead. Moving through the air in a risk-free manner against Denver in 2011 basically stopped at halftime of the SD game.
Who knows...maybe they hated Orton or something (lol). Or, maybe the benching of Orton reminded everyone of accountability...Fox wasn't just going to sit there and let his team get outgained 700-200 or whatever. The de-evolution to dismal on both sides of the ball stopped when Orton sat. Not suggesting Tebow is a middle linebacker too...but to the degree there is synergy...and to the degree that intensity has picked up from where it had sunk...it can at least be part of the discussion. A complicated discussion that isn't answered by "-0.18" or eschewing skill sets and strategies from a blimp view that sees most games as coin flips.
"If the league stops issuing wins to only QBs, then Tebow wouldn't be the topic."
I totally agree. They also spend a lot of time discussing character and religion (two things that also have little to do with any on-field results).
There are allot of unique things going on with the Broncos this season most important of which being the effect of a new coaching staff having limited time with the team due to the lockout, the defeatist atmosphere of the broncos stadium stemming from all the drama last season (4-12, video tapping scandal etc), and then the horrific start to this season. Sure the scores (excluding the packers game) were within a touchdown but upon inspection of the numbers of the games and watching the play of the players, yes the score may have been close here was a team that had given up on itself and to an even larger extent so had their fans. Then insert Tim Tebow, in every statistic ever crafted to define a passable let alone decent quarterback he fails yet, the Broncos are 5-1 with him at the helm. Now yes there are 53 players on a roster but where was these other players earlier in the year? Why now do they have such a higher level of play? Its simple, regardless of how bad Tebow's play is (and i think we can all agree its atrocious) his Teammates believe in him and ultimately that has more of an impact than any statistic can really measure. I mean statistically Cam Newton should be the one receiving this high level of praise yet despite his ability his team hasn't taken to him as a leader as much as the Broncos have with Tebow. As anyone that has played football can attest confidence and trust in your quarterback is paramount and without it winning becomes almost impossible. Just look at the colts or the frustration thats going to continue to grow in Chicago. Yes the quarterback is just one player but truthfully he's the most important player on the field. No matter how good a receiver, running back, o-line, or even a defense they can't really do it without a QB to, at the very least, manage a game. But I digress... Despite Tebow's numbers he doesn't turn the ball over and he inspires his team especially when it truly counts.