College basketball has always seemed to be associated with team ratings. From RPI to more advanced systems such as Jeff Sagarin’s, Ken Massey’s and Ken Pomeroy’s (the “KenPom” ratings), college hoops has an abundance of quality team ratings.
In 2011, ESPN joined the party and created its College Basketball Power Index, or BPI, after Jay Bilas and other ESPN analysts requested an alternative to RPI. Defined by a few unique characteristics, BPI was originally designed to rate college basketball teams and identify the ones most deserving of making the NCAA Tournament.
BPI has performed well in its five years, but like any good ratings system, it can be improved. With that in mind, ESPN’s Sports Analytics Team spent the summer revamping BPI to make it more predictive than before. The team split “old BPI” into a forward-looking power rating (still BPI) and a backward-looking résumé rating (Strength of Record) to better identify the best and most deserving teams in the country.
Below is everything you need to know about ESPN’s new and improved college basketball metrics heading into this season.
What is BPI?
BPI is a predictive rating system for college basketball that's designed to measure team strength and project performance going forward. In the simplest sense, BPI is a power rating that can be used to determine how much better one team is than another. If Las Vegas sports books ever published the power rankings they use to set their betting lines, they would likely look similar to BPI.
RPI has been the go-to system for the NCAA selection committee because of its simplicity, but it fails to capture opponents’ true strength, margin of victory or other predictive factors relating to the difficulty of the game. BPI accounts for all of those factors and more.
It’s important to note that BPI is not a predictor of which teams will be in the NCAA Tournament (that’s what Joe Lunardi is for) or even the ones that deserve to be in the Big Dance (that’s what Strength of Record is for). BPI is best-used to determine a team’s chance to win individual games or group of games during the season.
What goes into BPI?
Each team's BPI rating represents its projected point differential against an average Division I team on a neutral court. The rating is composed of a predicted offensive and defensive rating, which is on the same net-points scale.
In the preseason, BPI’s offensive and defensive ratings are composed of four main variables:
1. Coach’s past performance: This is captured by the coach’s average adjusted offensive and defensive rating (points per 100 possessions) since 2007-08 (including games at other schools). A first-time head coach gets a replacement-level rating.
2. Recruiting rankings: The average recruiting grade of the incoming class (compiled by 247 Sports) from four recruiting services (ESPN, Scouts, Rivals and 247 Sports). The number of five-star recruits joining a program is an additional input because of the singular impact they can have.
3. Current roster returning: The minutes played in each previous season for returning players, including transfers. Each player’s highest percentage of minutes played in a season during his career is used to help account for injuries and other setbacks.
4. Performance of those returning players: Opponent-adjusted offensive and defensive rating for returning players.
To simplify, preseason BPI asks: 1.) What percentage of minutes are returning? 2.) Are those returning minutes any good? 3) How will the team fill the non-returning minutes? 4.) What is the coach’s track record?
These preseason components hold different weights depending on how many players are returning, but they are combined in a Bayesian hierarchical model to produce a team’s preseason BPI.
In the past, BPI was released in mid-December after enough games were played for the data to converge. With the newly-developed preseason ratings, BPI’s projections are now available from day one and will help with prediction accuracy going forward. Preseason BPI is meant to predict where a team’s BPI rating will be at the end of the year, and seven of the last nine NCAA champions have ranked in the top six in preseason BPI (running the model retroactively on past seasons), with the 2011 and 2014 Connecticut championships representing the outliers.
As the season progresses and more is learned about each team, preseason ratings hold less weight. The preseason rating, however, never fully disappears -- it holds predictive power.
Performance isn’t determined by a team’s win-loss record. Like many advanced systems, BPI accounts for how a team won its games, adjusting for pace, the strength of opponents faced, travel and rest. BPI accounts for outlier performances by down-weighting a team’s most extreme results. As a team plays better or worse than expected, or when its past opponents appear stronger or weaker, its BPI rating moves up or down.
What’s accounted for in its game predictions?
We tend to focus on a team’s rating on the 1-351 list, but BPI’s real goal is to accurately predict games. Its game-prediction model has some unique features and will serve as the basis for BPI’s season projections, strength of schedule rankings and Strength of Record calculations.
Like most game predictions, BPI accounts for team strength, opponent strength and game site. BPI also factors in the number of days’ rest for each team, difference in distance traveled from home and high-altitude effects. Though the altitude affects only a few teams, it has been found to be predictive in those extreme cases. Similarly, most home-court advantages are captured in the game site variable, but a cross-country trip to an opponent’s home court is not treated the same as any other road game. This is especially useful for neutral-site games, in which the closer team will tend to have an advantage in the rooting of the crowd.
With the game predictions, BPI produces a team’s percentage to win any game. By adding in pace and efficiency components, BPI can also project a team’s margin of victory.
Each team’s schedule is simulated 10,000 times to produce season-level outcomes such as each team’s chance to win a share of its conference title or its projected strength of schedule.
BPI’s strength of schedule
Strength of schedule is more important in college basketball than arguably any other sport. With 351 teams and 32 Division I conferences, it is necessary to measure a team’s overall and nonconference schedule difficulty to evaluate a team’s résumé.
Unlike RPI, which measures the strength of schedule purely by opponent W-L record, BPI’s strength of schedule factors in every variable that is included in its game predictions. Most notably, an 11-2 team from the ACC is not treated the same as one from a mid-major conference, and the site of the game matters. Winning on the road in college basketball is tough, and BPI’s SOS rankings reflect that.
To formulate BPI’s SOS rankings, each team’s schedule is run from the perspective of a back-end Top 25 team (about the 25th-ranked team in Division I). The teams with the toughest schedules will be the ones with the lowest expected winning percentage from the perspective of that team. For example, last season Virginia played the toughest schedule, meaning that if a borderline top-25 team played every single Division I team’s full schedule (accounting for all of the variables in the game predictions) it would have been expected to do the worst against Virginia’s slate of games.
Strength of Record
ESPN's Strength of Record takes strength of schedule a step further by accounting for how a team actually did against its schedule. Unlike BPI, which accounts for how the game was won, Strength of Record simply cares about the difficulty of a team’s schedule and the result (win or loss).
For example, last season, Kansas ranked first in Strength of Record entering the NCAA Tournament, and a typical Top 25 team would have had less than a 1 percent chance to go 30-4 against the Jayhawks’ schedule.
Strength of Record answers the question of which teams deserve to make the NCAA Tournament based on their body of work. It correlates more closely with the actual committee rankings and seeding than BPI, but Strength of Record is far less accurate when making predictions.
Over the past five seasons, 94 percent of teams that SOR deemed deserving to make the tournament ended up making the field of 68.
BPI’s accuracy
The ultimate goal of the newly-improved BPI is to get its game and season projections right. It has done a pretty good job with that over the years, with the BPI favorite winning 75.6 percent of games since 2007-08. That compares favorably to other systems such as KenPom, Massey and TeamRankings.com.
As can be seen in the chart on the right, BPI is well calibrated. When BPI gave a team between a 50 and 60 percent chance to win, those teams actually won 55.8 percent of the time. As with any projection system, if the BPI favorite loses it doesn’t mean the system was wrong. If a team has a 60 percent chance to win, it is expected to lose 40 percent of the time.
BPI’s effectiveness projecting games has translated to the NCAA Tournament. No, BPI hasn’t predicted every upset, but based on the percentage chance it gave teams to advance to each round, this newly improved BPI rating has lower error rates than most other systems over the last five years.
BPI isn’t perfect. There are teams and projections that won’t always make sense. Still, we are confident that BPI will be one of the most (if not the most) accurate systems out there for the upcoming college basketball season.