The most recent NCAA News talks about perhaps the single most important facet of the NCAA championships selection process: the data.
In recent years, the NCAA’s selection process at the Division III level has been, at times, laughable. How else do you explain tournaments where the brackets keep changing after they’ve been released, where the number of teams in each pool needs correction, and the regional rankings are based on incorrect records?
This past offseason, the Capital Athletic Conference proposed a way to end all of this embarrassment: Release the data.
We here at D3sports.com know it is not easy to wrangle the amount of data that we and the selection committee have to deal with. There are more than 11,000 basketball games in a Division III season and while we here at D3sports.com know the rules as to what is a regional game and what is not, not all of the schools do, and everyone with access to the system can change the status of their own games, not to mention report scores, sometimes incorrectly.
However, the beauty of the system is that all of you can see the data and suggest corrections. And you do, frequently.
The NCAA keeps all of this hidden. Not just from the fans, not just from us, but even from the coaches and the schools. They can’t see their OWP or OOWP without coming to our site, or their “official” regional record unless they happen to get into the regional rankings. Only if you are on a regional committee do you get a login to view this information.
So when the CAC suggested opening the data for all to see, we saw it as a great thing.
The NCAA did not. They threw up roadblocks to this request, claiming that it would take six figures worth of budget to allow everyone access, that it would require training and other things that would make it unappealing to the membership. Never mind that they already have a system in place to give some people access to it. It might require more processing power to expand that to all 800 schools, but most of the work involved is in the generation of user accounts.
The benefit is that everyone who creates a schedule in any sport would have the opportunity to be more educated as to how that schedule affects the NCAA Tournament selection process. You’d have instant access to your opponents’ winning percentage and their opponents’ winning percentage. Coaches would see the proverbial man behind the curtain.
Eventually, of course, the proposal was withdrawn.
Our source at the NCAA convention last month told us, “There was a look of distinct relief on the faces of those on the dais.”
Now, admittedly, they have gotten a little better this season, at least in basketball. But how are we to know everything is correct?
Responding directly to the concern about errors, it discussed improving the score-reporting program’s current ability to flag conflicting information submitted by institutions, by adding a function that automatically would trigger e-mails to regional advisory committee (RAC) members and institutional representatives when such errors occur.
Sponsors of Proposal 8 asserted that ranking and selection decisions have been made in the past without resolving such conflicts in data. The Championships Committee wants to clear up such errors before each ranking of teams by a sports committee.
Read the article for yourself. It’s good they are trying to fix things, and indeed, this year’s regional rankings have been based on far more accurate records than in recent years.
But it’s not even close to being enough. The selection process is difficult enough as it is. Let’s at least make sure the right data are being used.
Just this week, an adjustment of 3 wins, formerly considered out-of-region, was made to the Scranton women’s record, which probably affected this week’s Mid-Atlantic ranking and future tournament selection and hosting. Yes, let the choices be made using correct data.
They also have problems with the “distance” determination. I don’t know if it has been solved, but at one point the Muhlenberg-Marymount game was NOT considered a regional game. Here is what is funny about that… if you go to the “distance” website and put in Muhlenberg and Marymount in that order, you get a distance of 201 miles – not a regional game. If you do that in reverse, put Marymount than Muhlenberg – you get a distance of 197 miles – that is a regional game. So which is it? And why is this important? If it is NOT considered a regional game, Marymount would have one major contributor to their at-large bid consideration off the table – a win over a highly regionally ranked team.
Just something else that needs (or needed) to be fixed.
This issue is a failure by the NCAA to hold to their espoused ideals.
Yes, that is a strong statement, but look at the principles that this situation.
“Best practices” As we have seen in reviewing the various selection processes, one committee doesn’t adopt what has worked in another sport. Who in the NCAA administration is holding onto their “silo” and not adopting best practices of another sport. A simple example of this is seen in the Handbooks. There are numerous variations in the Handbooks. Some lists of schools are by region. Some conferences are listed by Region. Other handbooks list the conferences in the various regions. Some handbooks do not even list independent colleges or members of “Pool B” conferences that are competing in the various sports. There are errors in placing schools in the correct regions. It almost seems that there is no central collaboration in maintaining the database, but rather each sport maintains a database that doesn’t interact with other databases in the same organization.
“An environment that is opening to learning and improvement.” The organization is reluctant to adopt new techniques that arise outside of the organization. We have seen the improvements that this web site has added in providing information to its “customers”, the fans. The NCAA has internal customers as well, the coaches and AD’s. For a school not to have ready access to vital data such as OWP/OOWP is just not consistent with the ideals of the organization.
The frustrating thing about the data is that the NCAA puts out guidelines by which it will conduct the championship. An open and welcoming attitude to improve the processes would see the NCAA putting forth the first draft of data and invite its customers and “co-users” to “proofread”. (The customers want a successful playoff process as well.) With the tens of thousands of data points that are entered by hundreds of individuals with varying levels of expertise, numerous errors are bound to arise. The process is most important. The process must be executed to best degree possible. These sites (and the references in the blog) show that the NCAA has not learned this over the last several seasons.
I was saddened to see the “data” amendment withdrawn from the vote back in January 2009. The “look of relief” was the telling feature.
I hope that the interested parties will push this again. Getting it right for the sake of the student-athlete is the reason that we are doing this.
Pingback: D3hoops.com Daily Dose » Blog Archive » Fact-check the NCAA