Process vs. outcome, and why metrics can’t make it on fourth down

"You can say a lot of things about me as a coach, OK? I'm sure you do, and so do a lot of other people. But I'm just telling you something -- one thing I'm not, is scared. Are we gonna go for it on fourth down every time we have it at our own 20-yard line? No, I'm not saying that. But I'm not afraid to go for it if you guys give me the confidence. And that's the way I felt about this yesterday. I felt better about us going for it than I did about giving them the ball back.

"But I'm telling you -- if we call this, you'd better [bleeping] get it."

That was New England Patriots head coach Bill Belichick, the day after his team beat the Atlanta Falcons, 26-10, in Week 3 of the 2009 season. In the third quarter, Belichick called for a handoff to Sammy Morris on fourth-and-1 from the New England 24-yard line, a strategy that could have blown up in Belichick's face had Morris not converted the first down. Morris gained 2 yards on the play, Belichick was a genius, and the crisis was averted.

About two months later, the Patriots were facing the Indianapolis Colts, when Belichick seemingly lost his mind -- the hubris of this man!!! -- and tried to convert a fourth-and-2 from his own 28-yard line with a little over two minutes left in the game. The Tom Brady pass to running back Kevin Faulk was true, but Colts defender Melvin Bullitt stopped Faulk short. The Colts scored a last-second touchdown on their final drive, and won, 35-34. Everyone in the world had an opinion on Belichick's decision, and most of them were negative.

It was a season in which the Patriots fell short of the Super Bowl -- the New Orleans Saints beat the Colts in the NFL championship that year, in part due to a successful onside kick called by head coach Sean Payton to start the second half. Due to the proper execution on one risky play, Payton managed to straddle the fine line between clever and stupid, as Nigel Tufnel might say.

In truth, when you ask the NFL's decision-makers about risky moves, it's not about the difference between genius and idiocy; it's about the divide between process and outcome.

Aaron Schatz of Football Outsiders started studying these trends long before Belichick went for it and failed, and Payton went for it and succeeded. As he recently wrote on Deadspin, he's analyzed NFL risk management approximately "eleventy-billion times." He most recently did so in response to the gambit taken by Carolina Panthers head coach Ron Rivera, who chose to punt from the 50-yard line last Sunday against the Atlanta Falcons with a 28-27 lead, and 1:09 left the game. The ball was downed at Atlanta's 1-yard line, and if you could have stopped the game there, Rivera would have been a genius. However, the Falcons drove down the field with no timeouts, and Matt Bryant booted a winning 40-yard field goal with five seconds left.

Schatz on the decision:

The old school will say that Rivera made the right decision because rookie Brad Nortman launched a spectacular punt. The Panthers downed it on the one-yard line; the only way the punt would have been better was if the Atlanta returner had fumbled it away. Because of that great punt, Rivera's decision didn't actually end up hurting Carolina's win probability—at least according to those league-wide statistical models. (Remember, personnel matters: Carolina was still stuck isolating [safety Haruki] Nakamura in deep coverage against Roddy White, one of the five best receivers in the league.)

But you don't judge decisions based on results; you judge them based on process. When Rivera made the decision, he didn't have a magical crystal ball that told him Brad Nortman was going to land the punt on the one-yard line. Nortman easily could have put the ball in the end zone for a touchback—just like he did when punting to New Orleans in a similar situation near the end of Carolina's Week 2 win.

And not to put too fine a point on it, but Atlanta won anyway.

"It comes down to this," Panthers quarterback Cam Newton, who fumbled on the play before the punt,  told me on Wednesday. "In any franchise, owners own, coaches coach, and players play. As a player, I'm supposed to do what my coaches tell me to do. He felt that he wanted to kick the punt, and I was all for it, and that's what we did. I shouldn't have fumbled to put our team in that type of situation to begin with. If I wouldn't have fumbled we get a first down and that game is won. Any way you look at it, it could've been a lot of things that went our way at the end, but they didn't. Coach Rivera always says, 'in a game where you play good enough to win, you play just as good enough to lose.'"

Rivera told me that each conversion opportunity is different, and that probability is a subliminal factor at best. "It's rule of thumb," he said of the punt in question. "If you're at the 50-yard line and you get [the first down], it's a great decision. If you don't get it, all they have to do is go 35 yards and [kick the field goal]. And excuse the expression, but it's dumbassed.

"We make light of it, but think about this -- what happened in the Super Bowl three years ago with New Orleans vs. Indianapolis? They onside-kicked it, but what would have happened if the Colts recovered, and scored the very next series? Now, everyone's going, 'Oh, why would you do that?' It's a calculated risk, and if you feel it's what you've got to do, great. But remember, if it doesn't work, everyone says, 'Oh, by punting, you decrease your opportunity to win.' Yeah, but guess what? If I go for it on fourth down and I don't make it, I decrease my opportunities to win, too, don't I?"

So the metrics should tell you, I said somewhat flippantly, that if the other team is going to go 99 yards down the field and win the game, you shouldn't punt from midfield. Where was THAT metric when coach Rivera needed it, I wondered?

"That's what my thought was," he said. "If we kick the ball inside, and they have to go 99 yards to score a touchdown ... we have to kick it and take our chances. That's the decision I made at the time, and I'd probably do it all over again. It's one of those things. I see all these statistics coming out, and we have to be careful. What happens is, for those other 160 plays in the game, what do the statistics tell you then?"

And, I hypothesized to Rivera, if a coach starts to focus on the outcome as opposed to the process -- if he puts himself at the schematic mercy of the things he can't control -- that's where he gets lost.

"I think so -- I mean, you've got to look at the whole thing. A couple years ago, there was another coach who went for it against another team [referring to Belichick against the Colts] and didn't make it, and the inverse happened. Everyone says "Oh, don't forget what happened with coach Belichick -- he tried to go for it, and he didn't get it.' You give the ball to Peyton Manning in that spot, and Indianapolis wins.

"So, it's one of those things. It's a great decision if it works, and I guess that's my point, I guess if we had punted, and they had to go 99 yards to beat us, and we'd stopped them, it would have been a great decision, I guess."

Pete Carroll, whose Seahawks play the Panthers this Sunday, told me of his own risk-management debacle. "The best one I hate to remember is in the USC national championship game against Texas. It was fourth-and-1 1/2 at about the 38-yard line. We ran the same play we had been successful with 18 straight times in the season, the back-side end got in, knocked us down, we fell short, and we didn't make it. There were a number of reasons to go for it: One, we were really good at it. Two, if we make the first down, we take a knee and win the game. We didn't want them to get the ball back. All of those factors go into it.

"When coach Belichick went for it, he knew that he [would have been giving] the ball back to Peyton Manning. So, let's win the game here, instead of trying to win the game over there. There are a lot of things that are going on -- the flow of the game and all that."

Process in the game, outcome after the fact. We can use all the metrics in the world to try and forecast what will and should happen, but as with most things, trying to close the gap on that great divide is like saying you can build a bridge out of thin air. Saying that it can hypothetically be done is one thing -- putting your neck on the line and walking that imaginary bridge is quite another.