For Seattle Kraken, Goalie Evaluation Is a Tricky Analytics Exercise

by Josh Horton | Dec 15, 2020 | Kraken, NHL Expansion Draft, NHLSeattle

One of the important decisions for the immediate, and perhaps the long-standing future of the Seattle Kraken, is what it decides to do at goalie during the Expansion Draft. Sound Of Hockey’s Darren Brown speculated on which goalies the Kraken may be linked to recently. Based on Seattle devoting a great number of front-office hires to assemble a robust analytics team, a statistical approach will almost certainly be considered when choosing any player in the Expansion Draft, including goaltenders. But for the Seattle Kraken, what goalie analytics even exist?

With a deeper breadth of data available, the Kraken’s front office will have a more complete data set than the public. But that doesn’t mean the public can’t take a look at what may be available through an analytical lens. 

In a way, goaltenders are akin to starting pitchers in baseball in how they’re evaluated. Some people evaluate goalies and pitchers solely on wins. No matter how many goals a netminder gives up, a team win is all that matters.

But any person that thinks the game with a crumble of analytics will tell you this stat is not even emblematic of a goaltender’s performance. This is a very old-school approach to thinking about sports and especially hockey where there are so many variables that go into the outcome of a game. 

The same goes for goals-against average (GAA), which is a slightly more sophisticated stat, but not by much. It’s similar to ERA (earned-run average) for pitchers in baseball, but really, it’s even worse. The goalie isn’t solely responsible for every single goal a team gives up, because not every shot is created equal; even the best NHL goaltenders won’t perform well if they’re facing many high-danger chances every night. 

(Say it with us, folks: “Goals Against Average is a Team Stat!”)

Things get slightly more specific with save percentage, which provides a more comprehensive look at a goaltender’s efficacy, but there’s still quite a bit missing. It’s a stat that requires a large sample size to have substantial value and can be misleading because not all saves carry the same degree of difficulty. Also, some teams allow a lot of shots and some teams don’t allow many shots, which really influences percentages especially when dealing with small sample sizes. .

Some stats go beyond the “traditional” metrics that aren’t all that difficult to understand.

“Advanced” Goaltender Stats to Consider:

• First, there’s GSAA, which stands for Goals Saved Above Average. This stat takes into account the league’s average save percentage and then applies it to the actual number of shots a goaltender has saved. In theory, it demonstrates how many goals a netminder would save compared to a replacement-level goaltender – for baseball fans, this stat is conceptually similar to the WAR (Wins Above Replacement) stat in baseball. This stat has its flaws, however, because it lacks some key context and assumes every shot a goaltender saves is equal, which we know isn’t accurate, and it benefits goaltenders with the most games played. Rating this stat out per 60 minutes (the length of an NHL game) is the best way to determine how a goalie is performing on a game-to-game basis. 

• There is also GSAx from the website “Evolving Hockey”, which stands for Goals Saved Above Expected and provides a bit more uniformity and specificity. It draws from Evolving Hockey’s expected goal model, so it takes into account how difficult, in theory, a shot was that a goaltender saved. This stat is also best when rated out per 60 minutes. 

• dFSv%, a stat also found on Evolving Hockey that stands for “Difference (or delta) in Fenwick Save percentage,” is another good one to use. Fenwick, for those that aren’t aware, is a stat that is essentially a fancier way of saying “unblocked shot.” Evolving Hockey’s expected goal model calculates an expected save percentage and demonstrates the difference. 

• HDSV%, MDSV%, and LDSV% – which stand for high-danger, medium-danger, and low-danger save percentage, respectively – are also useful stats. They measure exactly what they sound like. High-danger shots have an expected shooting percentage of over nine percent, medium-danger shots have an expected shooting percentage between three and nine percent, and low-danger shots have an expected shooting percentage below three percent.  

• Quality starts, a stat available by Hockey-Reference.com, count how many times a goalie posts a save percentage in a game that’s above the league average for save percentage. A quality start can also be earned by saving over 88.5 percent of shots in a game with 20 or fewer shots on goal. When a goaltender secures a quality start, their team has about a 75 percent chance of winning, according to HockeyReference. 

• Quality Starts % (QS%) is also available and is a great barometer into how a specific goalie is performing. A quality start percentage above 53 percent is considered above average, over 60 percent is considered excellent, and under 50 percent is considered below average. 

Here’s how many of the goalie targets most connected to Seattle stack up in these metrics. 

Maybe somewhat surprisingly, players like Arizona’s Antti Raanta and Adin Hill and Montreal’s Jake Allen rate out favorably, while more well-known goaltenders like Vancouver’s Braden Holtby and Ottawa’s Matt Murray do not.

But unfortunately, goaltender analysis — at least currently — is steps behind the analytical data that we have for players, and it’s not always fair to compare goalies to one another.

The Wild, Wild West of Analytics

The issue isn’t that there aren’t bright people evaluating goaltenders.

There’s just a dearth of public and, more importantly, reliable data to draw from.

The only publicly available data is via the National Hockey League, and sometimes, there’s missing context, especially when it comes to goaltending. 

“We have to be very, very careful with goaltending analytics,” said Alison Lukan, a prominent hockey analytics writer who currently writes for 1st Ohio Battery, a Columbus Blue Jackets’ site. “It’s probably the weakest part of the game in terms of us understanding it and properly evaluating it.” 

“A lot of the publicly available data is wildly inconsistent when it comes to goaltending,” said Catherine Silverman, a hockey writer who is well-known for her goaltender analysis. “… The save percentages, the high-danger save percentages, the stuff like that, is a little suspect at times.”

A lot of the popular analytics websites, such as Evolving Hockey and Natural Stat Trick, scrape data from the league website, which isn’t as precise as the private data the 31 current NHL teams are using to draw conclusions. 

This is highly problematic considering how many metrics hinge on where on the ice a shot on goal was taken and how the shot was created. 

“I always tell people that the first thing to know is that the data is going to be a little suspect at times,” Silverman said. “We’re in the infancy of goaltending analytics because it’s really the last position to be delved into from an analytical level at a deep level. It’s still very behind what you get from forwards and defenders.

What’s more, the lion’s share of goaltender analytics doesn’t take into account these key questions: Was the goalie screened? Was the shot off a rebound? Was the goalie moving laterally and what was their depth in net? 

“You can still look at that data, flawed or not, to look at an individual goaltender throughout the course of a season,” Silverman said. “See how their numbers are going up and down, see how much they’re fluctuating, see what they’re looking at year-to-year as long as they’re with the same team. You can use it for personal growth and improvement and you can use it for consistency, but it’s hard to use it to compare players because the data coming from each arena isn’t always consistent.”

There are many private companies, such as Clearsite or SportLogic, that can evaluate those aforementioned considerations, but that data is typically only available for the 31 franchises in the league.

What’s Next 

Puck and player tracking is expected to roll out any year now, but it’s unclear how much will be made publicly available by the NHL. 

That’s the only way to remove the “noise” from many of the goaltending stats people are referencing and compare goalies to each other fairly. 

“Until we get standardized puck-tracking, whether it’s through chips in the pucks or technology in all the rinks, goaltending analytics can’t take the step forward that it needs to,” Silverman said. “We truly need that locational data for shots, we need to know who is facing shots that are being blocked, who’s facing them while being screened, who’s facing them from crazy angles.” 

Right now, that sort of data is mostly being manually tracked, which is quite arduous and subject to the “eye-test.”  

Like any hockey statistic, goaltending analytics are best applied in harmony with live scouting. But for the common hockey fan, these aforementioned metrics can provide a better understanding of how effective goaltenders are through observations we aren’t able to see. 

Striking the perfect balance may determine how successful the Kraken are in 2021-22. 

Josh Horton is a freelance writer, former newspaper journalist, and former Western Hockey League writer for the Everett Herald and The Spokesman-Review (Spokane). He is NOT a juggler, nor is he a former professional baseball player. Follow him on Twitter @byjoshhorton.