MLB wants to use WAR to decide bonuses. The people who calculate it aren’t thrilled

Early last fall, an MLB representative reached out to Baseball Reference founder Sean Forman.

The ensuing conversation covered an issue that would months later be a significant story for both sides, the idea of potentially using Baseball Reference's well-known Wins Above Replacement stat as a way to determine the allocation of millions of dollars in bonus money for MLB's youngest players.

Forman, who wasn't a fan of the idea, described the call to Yahoo Sports as brief and said he presented some of his concerns to MLB. He said he assumed there would be some kind of follow-up if the idea was being seriously considered, but none came from either MLB or the MLB Players Association.

The next signal from MLB would come in January, via the media. Forman saw a tweet reporting MLB had proposed using WAR to allocate bonus money. He delivered his response:

Forman would soon receive endorsements from figureheads of two of the other major public outlets that calculate WAR, FanGraphs managing editor Meg Rowley and Baseball Prospectus editor-in-chief Craig Goldstein. He followed up his own tweet with a thread laying out six different concerns about the idea.

Since then, MLB's proposal and the MLBPA's proposal have come a bit more into focus. As Yahoo Sports' Hannah Keyser reported last week, MLB proposed using FanGraphs' version of WAR (known colloquially as fWAR), while the MLBPA suggested using the midpoint of Baseball Reference's and FanGraphs' number to allocate bonus pools for pre-arbitration players. Those players, most of whom don't get to negotiate raises through arbitration until they have reached three years of service time, would ordinarily make the minimum salary — the size of which has been another sticking point in negotiations.

Keyser reports the sides are still far apart on how much money should actually be given out. The MLBPA originally sought $105 million, and has now dropped that demand to $100 million, while MLB wants $10 million. That's a lot of distance to cross, but just the fact that they agree on the WAR bonus pool is a good indication the idea could be put in practice in the very near future.

Why don't WAR's calculators want MLB to use it?

To understand why people like Forman are apprehensive about their work being used to decide millions of dollars in salary, you first have to consider the basic nature of WAR. It's not a fully objective stat like, say, home runs, ERA or OPS. For a layperson, it's an incredibly complex and somewhat subjective calculation that Forman likened to calculating GDP in a phone call with Yahoo Sports.

Baseball Reference and FanGraphs aren't going to disagree on the top 30 players in stolen bases last season. They will, however, disagree on the top players in WAR, and that's where you might have some people, particularly players and agents, become bothered if the quirks of one version cost them a seven-figure payday.

The difference is particularly large when it comes to pitcher WAR. Baseball Reference's WAR (bWAR) focuses more on the outcomes of a game while FanGraphs is known for valuing the peripherals more, trying to separate a pitcher's true performance from as many complicating factors as possible, which can lead to a pitcher being ranked above another pitcher with a better ERA but worse FIP (Fielding Independent Pitching, an ERA estimator based on strikeout, walk and home run rates).

Using either version of WAR or both to determine bonuses could get messy for people with skin in the game.

"This assumes a precision to WAR that, I think people who think WAR is a really useful framework through which to understand baseball would be very quick to tell you, is not present," Rowley said on FanGraphs' popular "Effectively Wild" podcast. "So what happens if you are the 31st-most valuable pre-arb player and the difference in value between you and the 30th-most valuable is less than half a win? You're probably not the same, but you're within our margin for error on this."

To illustrate that lack of precision, consider last year's American League Cy Young Award winner Robbie Ray. Going off Baseball Reference WAR, he was the third-best pitcher in baseball with 6.7 WAR. For FanGraphs, he was 17th with only 3.9 WAR. Baseball Prospectus had him even lower at 3.5 WARP.

None of that is to say any of those numbers is objectively wrong. It just goes to show that if MLB decides the top 30 players should get more money, picking those players is not as easy as calling up a WAR leaderboard.

There's also the fact that WAR changes. No one in the future is going to deny that Juan Soto led the majors last year in on-base percentage (.465), but it's very possible Marcus Semien won't be considered 2021's bWAR leader for position players (7.3) in a few years, because there may be a change in how many runs he is credited with producing or how his defense is rated.

"A player could miss this bucket of bonus-eligible players one year and finish 27th, 28th in a recalculation," Forman said. "That’s not really my issue, but that’s something I could see happening probably with regularity if they go this route."

Baseball has changed to a remarkable degree over the last five years, from the increased prevalence of shifts, the appearance of the DH in the National League in 2020 (and possibly the future), the existence of Shohei Ohtani, the advent of the opener, other changes in pitcher usage and, of course, the fact that they've been recently playing with different rules in extra innings.

"If you’re a stat based on runs allowed, you then have to account for the fact that if they’re starting with a runner on second, that’s going to dramatically increase the chance they give up a run, so it’s not fair if we treat that like a normal inning," Forman said. "So we had to make a change for that."

What could MLB do differently?

Should MLB and the MLBPA use WAR as reported, Forman suggested the best way to sand down WAR's sharper edges would be to avoid a situation in which the difference between the No. 30 and No. 31 players amounts to a fortune.

"If you had a smooth system where the gradations were based on a smoother curve rather than a step function, where you’re either in or out, something like that could mitigate the issues of using a metric that possibly could change over time," Forman said. "People would probably not be quite as aggrieved losing $40,000 as they would be $800,000 or $1 million or whatever that difference would be."

As far as other stats that could be used, he suggested one alternative that was remarkably simple.

"I would say the number one indicator of player quality is probably playing time," Forman said. "I would probably be more inclined to go with something that relates to the amount a player has played, the amount they are used, because presumably the teams are trying to win and they’re playing better players rather than less good players.

"If somebody’s not playing, they’re not going to get WAR either."

And then there's the simple idea of paying someone for the work they've done, which has never been MLB commissioner Rob Manfred's forte, to soften the blow. When asked about receiving potential compensation for becoming such a major part of MLB's labor decision-making, Forman said he felt the idea was "appropriate and justified."

Obviously, none of these solutions are perfect. If MLB and all involved are going to use WAR for their new compensation system, they're going to have to accept a constantly shifting foundation controlled by small, independent companies they have little direct control over.

Some might be fine with that, but it probably won't stop players from airing their grievances on Twitter or agents emailing sites to consider that maybe the defense behind a pitcher wasn't calculated correctly. There's already enough vitriol over the very idea of WAR. Making it something that players have to consider when looking at their finances could easily ratchet up the disagreements.

There would be some validation for whichever WAR is used, though. What was an obscure statistical concept 20 years ago could become officially sanctioned in the basic goal of stats: looking at two players and deciding which one is better at baseball.

"There would probably be some benefit to us. ‘Hey, Major League Baseball is giving us a stamp of approval,’" Forman said. "The people who constantly say, ‘WAR, what is it good for,’ if the powers that be decide they’re going to decide salaries on this, it must have some value."