NHL Draft Efficiency Analysis
Evaluating NHL team draft performance from 2005–2017
Overview
This was my first data project, so I did it with the data that most excited me. I'm a huge hockey fan, and have been following the New York Rangers closely since 2007. From following the team for that long, I've noticed that the Rangers don't seem to be very savvy with drafting, and tend to acquire their impact players mainly via free agency signings; whereas other teams are seemingly able to consistently select impact players through the Entry Draft. I wanted to see if this difference in ability would actually show up in the data if we attempted to quantify it with a model.
To do this, I scraped draft result data from hockeydb.com from 2005 (the first draft after the post-lockout format change) through 2017 — far enough back to get real history, but recent enough that drafted players had time to actually develop into NHL contributors before the data was pulled.
Methodology
Measuring Player Effectiveness
For any model that I would use, I would need a metric to capture player effectiveness. The two main indicators of effectiveness available in my dataset were games played and points. Ideally I would have had more features available, such as advanced stats, but these were all that were available. The formula I ended up using was:
Performance Score = Games Played + (Points * 2.29)
This formula was informed from the data, and documented in this Jupyter notebook.
Since players drafted closer to 2005 had more time to accumulate games played and points, all players' performance scores were prorated to a capped window of 356 games to standardize the timeframe, which was also derived from the data and documented in the Jupyter notebook. This led to the final performance score formula:
Adjusted Performance Score = Performance Score * min(1, 356 / Games Played)
Meaning players who have played less than 356 games have their performance score untouched (356 / Games Played is greater than 1, so performance score is multiplied by 1), while players with more than 356 games played have their performance score scaled to the equivalent of 356 games.
Hindsight Draft Model
Initially, I wanted a model that would treat each draft as its own universe, and only compare players within the same draft class, as drafts are independent from one another. This led me to explore a hindsight draft order model, where for each draft, we get the hindsight draft order — simply the players from that draft ranked in order of their performance scores, giving us the draft order of how teams should have selected based on what we know today. Comparing the original draft order with the hindsight draft order, we can compute the differences in placement for each team and sum those up over all of the drafts to see how well teams under- or over-performed with their selections.
Draft Value Over Expected (DVOE) Model
This analysis uses a draft value over expected model to quantify draft effectiveness. To calculate a player's value, we take the difference of their performance score and the average of the performance scores of the other players who were selected at the same position in the draft. This gives a gauge of whether the team over- or under-performed relative to the value they were expected to extract at that position in the draft.
This does fail to account for the fact that different drafts have different available pools of players, and thus different relative strengths to one another, but as discussed above, this model swiftly handles many other problems introduced by trying to completely isolate drafts in their own context.
A team's draft performance value is calculated by taking the sum of their DVOE values over all of the drafts.
Tableau Dashboard
The Tableau dashboard was designed to give users a high-level glimpse into the draft profiles of all NHL teams within the 2005–2017 window.
The main chart, which serves as the navigation throughout the dashboard, is a horizontal bar chart that ranks the NHL teams in order of Average Draft Value Per Pick. The average was chosen over the total draft value in order to not reward teams for simply having more picks than other clubs, as the number of picks a team has isn't standardized — picks can be exchanged between teams via trades.
Clicking on a team's bar on the left chart populates the right half of the dashboard with various charts and tiles personalized with data from that team.
The top layer is 3 KPI tiles — Overall Rank, Early Round Rank (Rounds 1–3), and Hit Rate — which are color coded based on the percentile at which a team sits relative to the rest of the league. Colors range from green to yellow to red. Hit Rate is measured as the percentage of picks made by a team where the player's performance score was above the average score for the other players taken at that draft slot.
The middle layer can be toggled between two charts: Performance By Round and Performance By Year. Performance By Round is a dot plot showing average draft value across all years of the draft, grouped by round, color coded from green to yellow to red to capture league-wide percentile for that round. Performance By Year is a line graph plotting average draft value summed over each year's draft, to show historical performance trends per team. Clicking on a point in either chart filters the players shown in the table below for that team, allowing you to quickly view all players drafted by a team in a single year, or in a specific round across the dataset.
The bottom layer is the Players table, where all draft selections from the selected team are shown, with all fields from the database displayed and filterable. Users can sort by performance score, draft value, round, and draft year.
Analysis
Analysis for this project was conducted in this Jupyter notebook.
Analysis was split into two blanket investigation groups. First, I conducted statistical tests in order to cut through the noise of the data and see if any results were actually statistically significant. Second, I used k-means clustering to arrive at distinct drafting archetypes across the league for teams that revealed similar drafting profiles. This investigation also involved scraping actual team regular season and playoff performance data in order to test whether certain drafting archetypes produced quantifiable differences in performance results.
Draft Archetypes
To go beyond a single overall score per team, I used k-means clustering (k=6) on each team's round-by-round draft value profile — not just how much value a team generated, but where in the draft it tended to show up. This grouped the league into six distinct drafting archetypes, each with a different shape of strength and weakness across the seven rounds.
The Gold Standard
The two best-drafting organizations in the dataset by a significant margin. LA and Ottawa generate elite value across nearly every round, with exceptional depth in rounds 4, 5, and 7. Their hit rates sit well above the league average, and their overall output reflects sustained organizational competence rather than a single fortunate pick.
The Podium Powerhouses
Strong early-round performers anchored by exceptional Round 1 and Round 3 output. These organizations consistently extract value at the top of the draft and show a notable Round 3 edge, but efficiency drops sharply in rounds 4–7. Their overall average is slightly negative, meaning their early-round dominance is partially offset by below-average late-round returns.
The Second-Round Snipers
Defined almost entirely by a historically strong Round 2, which is the highest of any cluster by a wide margin. These teams consistently identify high-value players in the second round while posting below-average results in Round 1 and Round 3. The profile suggests a genuine scouting edge in a specific draft window rather than broad organizational strength.
The Efficient Moderns
A broadly positive cluster with above-average overall efficiency but inconsistent round-by-round output. Value is unevenly distributed, with strong returns in Round 1, Round 4, Round 6, and especially Round 7, offset by negative contributions in rounds 2, 3, and 5. The late-round surge is a defining trait, and their hit rate leads this tier.
The Late-Round Gamblers
A volatile cluster with highly uneven round-by-round performance. Early rounds are among the weakest in the dataset, particularly Round 1 and Round 3, but Round 6 is the highest of any cluster. The alternating positive and negative pattern across rounds points to inconsistency rather than a clear strategic identity. Overall value is negative, driven by persistent early-round drag.
The Efficiency Gap
The weakest cluster in the dataset, posting negative value across nearly every round. Early-round performance is particularly poor, with Round 2 being the worst of any cluster. The only marginal positive appears in Round 5, but it is not enough to meaningfully offset systemic underperformance across the rest of the draft. These organizations collectively represent the clearest examples of draft inefficiency in the league.
This is also where the question from the start of this write-up gets an answer. The Rangers land in the Late-Round Gamblers — inconsistent overall, and specifically weak in the early rounds, which carry the most value and the highest expectations. Individually, they rank 24th of 31 teams in average draft value. The data backs up the suspicion: drafting hasn't been a strength.
Key Findings
The "3.5% Variance" Discovery
A linear regression analysis revealed that team drafting efficiency (VOE) only accounts for 3.5% of the variance (R2 = 0.035) in total playoff series wins. This suggests that while a high-functioning draft provides a stable competitive "floor," championship "ceilings" are driven by high-variance factors like goaltending, trade acquisitions, and injury luck.
Isolating "True" Skill (Binomial Significance)
Using a Binomial Test against the league-wide hit rate, I identified that only three organizations showed scouting results statistically improbable to be the result of "noise":
- LA and Ottawa were significantly above the league average, proving sustained scouting excellence.
- Vancouver was significantly below, indicating systemic drafting inefficiency.
- For the remaining 28 teams, the results fell within the range of random variance, highlighting how difficult it is to maintain a "true" scouting edge over a 12-year window.
The Myth of "Getting Better" (LAG-1 Analysis)
To see if teams "learned" or improved their scouting processes over time, I applied a LAG-1 Autocorrelation test. Only St. Louis and Tampa Bay showed a statistically significant upward trend in efficiency. For the rest of the league, drafting performance was essentially "random" year-over-year, suggesting that organizational "lessons learned" are often offset by staff turnover and league-wide parity.
The Parity Paradox
Despite the diverse scouting behaviors identified in the clustering phase, five of the six archetypes were represented among the Top 10 most successful playoff teams. This proves there is no single "winning" scouting philosophy; rather, success is found by teams that effectively leverage the specific talent profiles (Early-Round stars vs. Late-Round depth) their system is designed to find.
Conclusion
The most striking finding of this investigation was the 3.5% reality. While the DVOE model successfully identified elite drafting systems — like the Kings and Senators — statistical regression showed that draft efficiency only accounts for a fraction of championship success (R2 = 0.035).
This leads to a critical insight for NHL analytics: drafting provides the floor, but the ceiling is dictated by variables that live outside the draft — trades, free agency, and high-variance luck. By separating process from outcome, this framework can't predict a Stanley Cup, but it can quantify which organizations are consistently making the most of the assets they're given, round by round.
This was my first data project, and the scale was modest — 13 drafts is a small, noisy sample, and I'd be lying if I said I wasn't hoping for a louder finding linking drafting to winning. But the absence of a strong association is itself a finding, and chasing a more dramatic story than the data supports would have been the wrong move.
If there's one thing I'd point to as the actual lesson of this project, though, it's not a stat about hockey — it's that my first attempt at a model (detailed above, for anyone curious) turned out to be structurally broken in a way no amount of tuning could fix, and the right move was to recognize that and rebuild rather than keep patching it. That instinct — build it, find where it actually breaks, and be willing to throw it out instead of duct-taping it — is the part of this project that's shown up in every one I've done since.