The NCAA has long faced criticism from both traditionalist coaches and stats-heavy analysts for using the Ratings Percentage Index (RPI) as a criteria when selecting teams for March Madness. That led the NCAA to create the NET ratings instead – and the early indications are that they could be even less popular than the RPI.
Ohio State (6-0) was ranked as the top team in the country according to the initial NET ratings released on Monday, a surprising result that wasn’t nearly the weirdest aspect of the first list released by the NCAA.
Initial Ratings Draw Widespread Criticism
The NET ratings are currently based on only five to seven games for most teams, so it’s understandable that a small sample size might lead to some slightly unusual results. Nobody is particularly upset that Gonzaga (6-0) is in fifth place, or that Kansas (5-0) is relatively low ranked at No. 11.
But some other inclusions have led to widespread derision from critics. Loyola Marymount (7-0) is ranked 10th in the country, while Belmont (5-0) is slotted in 12th place. Meanwhile, Kentucky – with just a single loss to Duke – is all the way down in 61st.
NET combines five different metrics in order to evaluate teams. These include strength of schedule, winning percentage, margin of victory, offensive and defensive efficiency, and a weighted strength of schedule based on the location of a team’s games (home, away, or neutral site).
Nate Silver: Worst Rankings Ever
One of the biggest critics of the system was Nate Silver of FiveThirtyEight.com. The data analyst has levied numerous criticisms against the NET ratings since they were released, at one point calling them “the worst rankings I’ve ever seen in any sport, ever.â€
These are the worst rankings I've ever seen in any sport, ever. NCAA needs to go completely back to the drawing board. https://t.co/UXKbrGyuP8
— Nate Silver (@NateSilver538) November 26, 2018
“It’s worse than RPI. Worse than the eye test,†Silver wrote on Tuesday. “It’s why you never want to design an algorithm by committee.â€
On the other hand, there have been a few voices urging fans and pundits to give the rankings time to sort themselves out as the sample size of games improves.
“I think it’s too soon to give a true and fair evaluation of any metric that is based entirely on data from this season,†analyst Kevin Pauga told ESPN. “A single result right now could range from 12 to 25 percent of a team’s resume, depending on how many games a team has played. By the end of the season, each result will constitute about three percent of a team’s resume.â€
But in a Tuesday tweet, Silver pointed out what he thinks is the largest issue with the NET system, one he says won’t be fixed just because more games are played.
“The big problem seems to be that only one of the five components…considers strength of schedule,†Silver wrote. “So it rewards [teams] with good records against very poor schedules. Probably won’t get much better as the year goes along.â€
Even if the NET ratings continue to spit out bizarre results for the rest of the season, college basketball fans may not have to panic. While the selection committee will use NET as part of its evaluation, it won’t be the only factor any more than RPI was in previous years – and members could quietly devalue it should it appear to be unreliable at the end of the season.
