Fielding Metrics, a Brief and Incomplete Study

Defensive statistics are considered to be the final frontier of the sabermetric movement.  While the statistics available these days paint a picture that is much clearer than the fuzziness of fielding percentage, there is still a lot of room for improvement.  Two of the best defensive metrics that are readily available right now are Tom Tango's Fans' Scouting Report and Ultimate Zone Rating.  Let's now look at this year's best defenders according to these two statistics, see if they pass the smell test, and compare them to the results of the Gold Glove winners.

By the Fans, for the Fans


Developed by Tom Tango, the Fans' Scouting Report (FSR) combines performance analysis and scouting observations.  FSR uses the combined wisdom of baseball fans, and the strength of crowdsourcing - One person's opinion on Chase Utley's defense is pretty worthless, 5 peoples' opinion is better, if you get 50 peoples' opinions on the subject you should have a pretty good idea of that players defensive prowess.  Every year (since 2003), Tango puts out the call on various saber-friendly blogs, and has fans evaluate players defensive abilities of players they have seen play at least 10 games that year. The fans are asked to evaluate specific defensive attributes of each player (speed, hands, arm strength, throwing accuracy, instincts, etc.) rather than give an overall gut feeling.  So, what you end up with is hundreds of people giving their opinions on individual players, and more often than not, it ends being a pretty accurate portrayal.  Pictured below are the best defenders in each league according to FSR.

In the Zone


Ultimate Zone Rating (UZR) is a statistic created by Mitchel Lichtman, which divides the playing field into 78 zones, of which 64 are used in the calculations.  Pitchers and catchers are not judged by this statistic, and infield fly balls, line drives and outfield foul balls are not included.  Play-by-play data is used to determine the percentage of balls in each zone that are turned into outs by a particular defender, and then the league average is subtracted from the player's out rate for that zone.  The remaining number then determines whether a player is above average (positive number) or below average (negative number).  This is then adjusted for park factors (it's a lot easier to man left field in Fenway than it is in Petco) and takes into account the particular situation of each at bat, and using run expectancy charts, is converted into theoretical runs saved or given up.


As with all statistics, sample size is important when trying to glean information, with UZR it is even more important.  UZR tends to fluctuate wildly from month to month, and even on a yearly basis we can see major changes.  Many people have said that a three year sample size is best when trying to get a true evaluation from UZR, so take these one-year numbers with a grain of salt.

Comparing these two images above to the Gold Glove winners (pictured below), we can see that UZR matches up to 8 of the 16 eventual winners whereas FSR only matched up to 7, even though UZR does not account for catchers.  Does this mean that UZR is a better defensive metric than FSR?  Probably not.  The Gold Glove has long been scoffed at as far as it's accuracy in naming the best defender in each league (Rafael Palmiero's 1999 award for 1B, when he only played 28 games there and spent the rest of the year as the DH would be Exhibit A).  And as I said before, there can be some goofy numbers with UZR from year to year and these numbers should be taken with a grain of salt - case in point David DeJesus.  I doubt DeJesus himself thinks he is the best defensive right fielder in the league, yet UZR has him as a +12 fielder in 2011 (though he was just below average for the first four years of his career).  With FSR, you can make arguments for other players being better than the "winners" from this year, but there isn't really anyone on that list that is really shocking, in my opinion.  It is also interesting to note that the four positions where FSR and UZR agree (AL 1B, NL LF, both 2B) all end up taking home the Gold Glove.


(In both images, * denotes #1 overall MLB ranking)

With the Gold Glove, it seems almost every year that, for the most part, the voters get it right.  But invariably, there are a couple of selections that are so strange they all but invalidate the award as a whole.  Most notably this year, the selections of Matt Kemp (12th overall ranking among NL CF according to FSR, -4.7 UZR) and Andre Ethier (16th overall ranking among NL RF according to FSR, +5.3 UZR) don't pass the smell test.  These two selections play into the stereotype that the Gold Glove voters pay more attention to batting stats, past performance and name recognition more so than fielding prowess.  The selection of Erick Aybar (7th overall ranking among AL SS, +1.2 UZR) seems strange as well, and personally, I think Ramirez and Escobar are much better choices.  But at least they didn't give it to Jeter again, and for that we can all be thankful.

x x x

Looming on the horizon right now is the advent of a technology that may finally put an end to the speculation that goes into determining defensive abilities.  Long considered to be the advanced stat Holy Grail, FIELDf/x is supposed to be implemented in all 30 MLB ballparks this season.  Developed by Sportvision (the same company that developed PITCHf/x), FIELDf/x uses high resolution cameras to plot every player on the field, their actions on every play and time stamps each frame.  No longer will we have to speculate on factors of range, positioning, route running and throws, we will now have actual data to back it up.  When this information will be made public is still uncertain.  There is an enormous amount of data that will have to be sorted through and cleaned up to get to the meaningful information (ie. balls in play), and only a few people have been given access to the test data to analyze it.  This will indeed be a very exciting breakthrough in baseball technology.  Being able to track hang time on fly balls, pop times from catchers, the time it takes to steal a base, etc. will only help to enhance the fan experience on TV broadcasts and interfaces like Gameday, as well as to help us more properly evaluate defensive abilities.  


for further reading by people much smarter than me:


FanGraphs UZR Primer
Tom Tango on FSR 
FIELDf/x