With the Mariners off today, I figure I should take some time to explain the Net Runs scoring system in greater detail. This is long, and despite its convoluted length, I don't expect this to make total, perfect sense of the whole system in one try. But it should get you closer to understanding how it works.
The Net Runs system takes every result from play by play data and assigns a run value, positive or negative, to an appropriate player, whether at the plate, on the mound, on defense or on the basepaths. Managers and the park can also receive runs when they factor into the outcome, as can pure luck. (In some unfortunate cases, the team's good luck could be one of their biggest run producers in a game!)
Every game references a park-specific run expectancy chart devised from 2005-2009 MLB average lines that are park adjusted to the home park. Park factors for hits, doubles, triples, home runs and walks are devised from ESPN's simple park adjustment equations over the five year period of 2005-2009 (or less for newer parks) and then used to adjust the 2005-2009 composite MLB average, before the revised numbers are run through Tango's Markov equations to determine run expectancy by base/out situation for each individual park. All of these are kept on a separate spreadsheet, and for each game I simply add the appropriate run expectancy chart to the scoring sheet.
(For games at brand new Target Field in Minnesota, games will use the park factors from pitcher friendly Oakland Coliseum, as meteorlogical wind patterns indicate the wind will typically blow in from right field after the first month of the season (the wind may blow out to center during the first month), and the weather patterns for the Twin Cities indicate cold early/late season games along with muggy mid-summer games, all friendly for preventing offense. Once we have sufficient data to determine otherwise, those numbers will be used to determine the true park adjusted run expectancy for Target Field.)
The batter is credited for any outcomes at the plate in his control. Hits, unintentional walks, strikeouts... but not errors and intentional walks. He does get credit for hit by pitches: If nothing else, he ought to get some credit for his pain :P
The pitcher is credited in the pitching column for:
- Walks and hit batters
- Strikeouts
- Wild pitches
- Balks
The other outcomes are where things get fuzzy. A pitcher only has so much control over the outcome of a batted ball, with the rest being up to his defense, the baserunners, the dimensions of the park and other assorted factors. However, as Dan Fox and Tango found years ago, each of the three types of batted ball outcomes (groundballs, flyballs, line drives) tend to go for hits a certain percentage of the time, with the power of those hits varying in kind.
I researched Retrosheet data to find the average run expectancy change for strikeouts, walks, stolen bases and times caught stealing, plus each of the three major batted ball types: Groundballs, Flyballs and Line Drives. From complete 2009 data I found that the league averaged the following RE change for each:
Groundballs: -0.088
Flyballs: -0.030
Line Drives: 0.324
Stolen Bases: 0.174
Caught Stealing: -0.434
Walks: 0.309
Strikeouts: -0.282
However, that's taken in a vacuum from play to play, and may not necessary reflect the true weight of these events. I mainly concerned myself with the first three values. I made sure to break down groundballs between regular groundballs and bunts, and to break down flyballs between pop flies (which per Fox are outs about 98% of the time) and outfield flies. I made a few observations:
- Bunts make up about 5.5% of groundballs, have nearly the same average net RE (-0.085 for 2009) and do not affect the average net RE at all. We're typically talking about 10-15 runs lost per team on top of an overall sample of 150-175 runs per team.
- Pop flies average about -0.285 runs per incidence, and per Retrosheet compose about 19% of all flyballs, a huge difference from data on Fangraphs that indicates the average on infield flies is closer to 10-11% since they only count infield pop flies whereas the Retrosheet data is for any pop fly fielded by an infielder, whether on the outfield grass or in the infield. Needless to say, pop flies dramatically influence the average net RE of a flyball.
- Pop flies aside, the net RE of flyballs from park to park in 2009 varies considerably, from +0.036 per outcome in Fenway Park (BOS) to -0.101 in Busch Stadium (STL). Flyballs are probably where much of the variance in offense between parks comes from.
- Though the variance wasn't as great, they definitely existed some variance in groundballs and line drives between parks. Groundball rates ranged from -0.127 per outcome in Petco Park (SDP) to -0.061 per outcome at Chase Field (ARI). Line drives ranged from +0.260 per outcome at Great American Ballpark (CIN) to +0.379 per outcome at Tropicana Field (TBR).
A perusal of player batted ball splits (via Baseball Reference) as I researched projections for 2010 showed that certain players had dramatically varied averages and slugging for the three batted outcomes. Without making this overlong post impossibly long with too many details, I'll briefly summarize....
- Faster players tended to have higher averages on groundballs while slower players tended to have lower averages... but there were plenty of exceptions. Victor Martinez was one player not known for his speed who managed relatively high averages on groundballs some years, and has a career .236 average on groundballs, close to the league average.
- Players tended to have dramatically different averages on flyballs, which didn't necessarily correlate to the player's home park. And, of course, the slugging percentages for flyballs were all over the place, typically varying upward or downward according to the particular player's power. Home runs were a factor in this but so were doubles, triples and pop flies.
- Line drive batting averages varied up or down seemingly at random, but tended to stay around the .675-.800 range, true to the league's 2005-2009 composite average of .732. Of course, isolated power for line drives varied slightly up or down from the league average, typically trending in line with the respective player's power.
These numbers were useful for projecting individual players' potential performances for 2010, they indicated that there was a lot of variance to batted balls, to the point where blanket-crediting the pitcher for outcomes as other incumbent systems do (WPA, RE24) wasn't necessarily fair. Thus I decided that the fairest approach would be to find an average run value for each batted ball outcome, credit the pitcher for that run value, and then credit the defense and other associated contributors for the difference.
This led to a epiphany, hardly a unique one but one of significance. If we were to score every game in a league with the Net Runs method, you could compile a total number of runs a defense contributes to their team's run prevention. Unlike other methods (UZR, PMR) that rely on a separate behind-closed-doors mapping process and a lock-and-key "Just Trust Us" value-approximation process to estimate a cumulative run value... this method could allow an observer to see the process in full and, like basic rate stats, see where the numbers came from.
In other words, unlike the incumbent defensive rating systems, you don't have to wait until later in the season or season's end for a proprietary service to release the numbers. One who is willing could watch a game and, like the final box score, provide an immediate number of defensive runs allowed or saved.
Is it the most perfect and accurate system for rating a player's defensive ability? I would never argue that there was such a thing as a perfect system for rating defense. All incumbent defensive measurement systems are subjective, basing their assessed run values on averages for plays made in a set of arbitrarily drawn and assigned zones... no more arbitrary than some of the basic rules in scoring Net Runs on defense. Is the logic for assigning those zones reasonable? Sure: It's based on aggregated totals through history. But it's also not an open process, nor an automatic process. And given UZR and PMR ratings can vary wildly from year to year without appreciable changes in a player's ability, the systems lack the consistency needed to cement them as definitive measures.
A reliable process for measuring value has a clear method for assigning value to a defensive play whose logic an outsider can follow. While it's hardly the best gauge of a player's talent, the OPS number is a perfect stat in that it can be easily compiled at once from freely available data, an observer can tell where you got the numbers from and it makes sense how the number came to be.
That's where Net Runs bridges the gap. While admittedly not a definitive judge of talent, it can assign a firm value to a player's defense in a given game, and over a season can give you a net total that will give you a solid value on what a player contributed with his hitting, defense, pitching and baserunning, a number that can be compared against his peers to give an idea of which players excel (or scuffle) at their respective positions.
It is admittedly not an easy process. Even if you compose a spreadsheet designed to streamline the process (as I did), it still takes time to go through the individual games and score them. Even with several shortcuts, the average game takes me 15 minutes to score from the game log and MLB Gameday data in its entirety. Other numbers can be easily compiled from cumulative or box score data via programmed scripts, but scripts can't easily evaluate play by play text and MLB Gameday to decide who in the infield deserves credit for letting a groundball get into LF for a single. You could take a page from Sean Smith's Total Zone numbers and just arbitrarily assign weighted values to every potential fielder, but the end result often gives undo credit to a lot of individuals. With older (lacking) play by play data, you don't have much of a choice, but with present day data there's no need to do that.
More later on the in-game and game log methodology. Believe it or not, the process can easily be done on your computer while watching, tracking or listening to the game. (You could feasibly do it live at the game, but you would need some reference charts, possibly a calculator, a good judgment of batted ball types from where you sit, to never at any point take your eyes off the action, and honestly I'm not sure I'd recommend it unless you're in a press box and have technology at your disposal).
No comments:
Post a Comment