By the Numbers: Extreme failure

You can find more from Michael Salfino at NESN

We're going to do something in this column that you'll rarely see an expert do – examine failure. By that I mean, predictions gone awry.

Failure is an unavoidable product of risk taking and making early calls on players, calls that in most cases vary significantly from general market view. The buy low and sell high strategy implicitly acknowledges the inevitability of failure by fixing a lower cost on it.

A recent book, "Dance with Chance" (Markridakis, Hogarth, Gaba), focuses on business but offers great insight into the role expert advice should play in decision making.

There are no formulas or tricks that guarantee success. No one can absorb your uncertainty and help you realize that dream of total control and predictive infallibility.

Demand from your experts state-of-the-art knowledge, independent thinking and due diligence in research as if every player discussed was their own. Even then, though, we are merely informing your judgments.

Fortunately, there are not too many predictions that I'd want to take back right now. But many involve underestimating the limits of predictive uncertainty. "Dance with Chance" advises business to look at the extreme ends of the historical data and then double it to get a firmer sense of the range of future outcomes. When there's a lot of historical data, you estimate uncertainty by multiplying the range of past outcomes by as low as 1.5.

But how do you perform multiples on the high and low end of players' stats? The book's advice clearly doesn't work. Good and bad past stats aren't monetary gains and losses that can be easily multiplied.

Baseball Prospectus attempts to address the limits of uncertainty with their PECOTA projections. But even a formula dedicated to quantifying the limits of uncertainty badly underestimates it. For example, Aaron Hill's(notes) 90th percentile of PECOTA performance this year was 12 homers. Ben Zobrist(notes) (a player I've been wise or cowardly enough to completely avoid) had a 90th percentile/upper range of 10 homers in 328 at bats (.470 slugging), even though he slugged .500 in limited time last year.

I don't know how to quantify uncertainty for the purposes of projecting baseball performance, but I guarantee that we must start with the premise that our view of its limits are far too unimaginative.

Let's look at the players we've gotten most wrong and see if we use our failure to inform future judgments. We'll do away with the recommendations this week and instead look at …

… What Went Wrong (So Far)

Aaron Hill, 2B, Blue Jays: Have to go back to the Yahoo! Fantasy Baseball Guide for this one. I was aligned with the Baseball Prospectus camp. And I should note that their 90th percentile slugging percentage (.451) is not as far off from actual (.477) as their homer projection seems likely to be. Hill only has nine doubles vs. the 47 he had in his 17-homer year. I should have looked at that doubles total as the best barometer of his power potential.

Raul Ibanez(notes), OF, Phillies: It wasn't park factors. The only thing worse than making a mistake is attributing it to the wrong reason. But lineup is an environment, too. He's old and now hurt. But that's no solace to me because I didn't argue that Ibanez would slow down due to injury. I simply said he wouldn't keep hitting as many homers – which he kept doing well after I wrote it.

Ian Snell(notes), P, Pirates: He was an early season velocity recommendation. It's down about a half-mile an hour. But Snell still is getting pounded. His FIP (fielding independent) ERA is 4.57, far too high to be as useful as I thought he could be. It's reasonable to wonder whether bad luck causes bad pitching or vice versa.

Matt Cain(notes), P, Giants: I said to sell him twice based first on K/BB ratio and then on FIP. Yes, his strand rate is 89 percent (average is 70 percent), but there's some chance he just dials it up with guys on base. I'd never bet on that. But I've discounted the possibility too much. Perhaps the more stuff you have, the luckier you may be.

Pablo Sandoval(notes), 3B, Giants: His batting average on balls in play (BABIP) generally has been much higher than normal. But yet I obsessed about his BABIP this year in putting a sell on him. I discounted the fact that Sandoval seems to be an exception to the rule in being able to maintain a much higher actual on-base plus slugging (OPS) than what's predicted.

Mike Jacobs(notes), DH, Royals: I said to hold his power just before he went a month (and counting) with one homer. Yes, Jacobs is an extreme fly-ball hitter and that's the necessary first step to hitting homers. But there's little evidence in his career that he's a good hitter, the far more important consideration.

Michael Salfino's work has appeared in USA Today's Sports Weekly, RotoWire, dozens of newspapers nationwide and most recently throughout Comcast SportsNet, including SNY.tv, for which he also analyzes the Mets and Yankees. He's been writing "Baseball by the Numbers" weekly since 2005.