As many college football fans follow recruiting and its relative importance to winning championships, the understanding that recruiting ratings (which largely determine star-ranking status) are a good approximation of team talent. This article is even more evidence that the rankings do a good job of identifying high school talent.
From my point of view, the NFL draft is the best measure of whether a player is talented or not. NFL teams spend a lot of money, time , and opportunity cost on each draft pick. As such, they have the most skin in the game and (presumably) relevant expertise in talent evaluation.
This article outlines the probability of getting drafted by using a more granular star-ranking (https://thefaircatch.com/2021/06/11/a-more-granular-view-of-star-rankings/), which is an expansion of the 247 Composite ratings. Furthermore, it highlights a possible ramification to the arbitrary cutoff of approximately 30 five-star players in each cycle.
Data
Recruiting class data and NFL draft data were obtained from http://www.collegefootballdata.com. These data sets spanned the years 2005 to 2021. There were 53,826 high school rated players and 2,307 records for drafted players.
Using a more granular breakdown of star-rankings provided insight as to where players draft propositions started to diverge.
In looking at the above scatterplot, a few things become noticeable. First, there is a visible cutoff at the 5.0 star-ranking in the upper left quadrant, which represents earlier-round draft picks and highly rated recruits. This is like because of the arbitrary 30-player limit to 5-star status. It pushes good players down into the high 4-star range.
There is also a clear diminishing presence of higher-rated players to the right of the vertical line (mean line for draft pick).
Below the mean line for star-ranking, things look fairly uniformed.
Probabilities
My hypothesis is that the arbitrary cutoff for 5-stars artificially divides them from 4-stars. The stacking up of data points along the 5.0 star range supports this belief. In looking at further breakdowns of the data, this hypothesis continues to be supported.
Taking all of the 53,826 observations against the NFL draft data and breaking it out by incremental star-rankings, the probabilities for each category getting drafted is shown in the bar graph above. To achieve even a 10% overall chance of getting drafted, a player needs to be a 4.2 star, or have a composite rating of 0.9087 at a minimum. To have a 20% chance of being drafted, a player should be a 4.9 star, which is a composite score of 0.9738.
Also visible here again, is the bottleneck right at the five-star cutoff, strengthening the case against the arbitrary 5-star cutoff.
Drafted by Round
Just getting drafted is a huge marker of success in my view. But getting drafted in early rounds is that much better.
In the bar graph above, the round drafted proportions are added. Once again, players at the 4.9 star rating are considerably more likely to be drafted in the first two rounds. Below, I broke these out by NFL draft round
Conclusion
It’s pretty clear that better players out of high school are more likely to get drafted, and more likely to get drafted earlier in the draft. However, the arbitrary ~30 player cutoff for 5-star designation strikes me as a marketing and hype gimmick. Unfortunately, this probably denies some very deserving players of the five-star status.
Based on what I’ve found, the 5-star designation should be awarded to any player with a composite rating score of ~0.9738, which would give them a 20% chance of being drafted.
Just looking at some of the data from http://www.collegefootballdata.com, I was surprised to find that UF’s offense is performing at a lower level than their defense overall
These boxplots show that UF’s offense is not doing well on 3rd down.
Looking at the overall performance, the below violin plots indicate there is a statistically signficant difference in EPA (p = 0.005) between the two sides, with the offense showing much higher variance:
With UF football unexpectedly struggling, there is considerable criticism of head coach Dan Mullen and starting quarterback Emory Jones (EJ). Not to mention the ever-present criticism of defensive coordinator Todd Grantham. However, one source of frustration among UF fans and college football analysts and pundits everywhere seems to be that Mullen continues to start Jones over freshman QB Anthony Richardson, or AR15.
Though this frustration often manifests itself as a slight against Jones, it may be more of a compliment to AR15. Using EPA data from CollegeFootballData.com I looked at how EJ, AR15, and Florida’s opposing QBs (OPP)have performed so far. I just used UF’s opponents as kind of a reference point so we aren’t just looking at EJ vs AR15 without some additional context.
The sample sizes for AR15 vs Jones and OPP are quite different. I was able to get 65 plays for AR, 255 for EJ, and 217 for OPP. Because I am lazy, I only used pass plays for opposing QBs but I used rushing and passing for Florida’s QBs. Sifting through every play and pulling out which rush play was by an opposing QB would take time and it isn’t that important to me. I just wanted some level of reference point. Keeping rushing plays for the UF QBs also helps bolster AR15s sample size. Plus, QB running is a big part of UF’s offensive philosophy with both EJ and AR15.
Just graphing the performance over time shows us something interesting. We can see that AR15 has a lot of plays above the upper bound (2 standard deviations above the group average) and few below the lower bound.
The dotted green lines are the upper and lower bounds. Black dotted line is the average. Using this as a threshold for Really Good and Really Bad, the simple probabilities (percentages) work out like so:
Player
Really Good
Really Bad
AR
11%
2%
EJ
2%
2%
OPP
1%
1%
The clear extreme score is the probability of AR15 having a Really Good play. EJ has largely performed at the same level as UF’s opposing QBs.
This can also be seen in a simple boxplot where the outliers are shown (but only at 1.5 standard deviations, above and below the interquartile range).
The boxplots show that EJ has quite a few negative outliers (dots at the bottom), while all of AR15’s outliers are at the top (good).
I’ll continue to update throughout the season. The Georgia game will be a great opportunity to see what AR15 can do against an elite defense, though he will surely need some help from his teammates. Once I get some more time, I’ll include just the SEC QBs to get a better reference point.
Using data on offensive linemen for recruiting classes from 2005 to 2021 (data: https://www.collegefootballdata.com/), I found a statistically significant difference between 3, 4, and 5 star categories (p < 0.001). A Shapiro-Wilk test indicated a non-normal distribution for weights (p < 0.001), therefore a Kruskal-Wallis test with pairwise comparison was conducted.
Median weight for 3-stars was 285. 4-stars was 295. 5-stars was 305. Apparently weight is a factor in the ratings process.
One of the main problems with the star-categorization is that, like most ordinal data, it doesn’t provide an idea as to the degree of separation between intra-star gradings. Generally speaking, we can use the individual ratings to find that, which usually works well. But that still doesn’t let us know exactly where within a particular star category a player currently sits. Taking the range of ratings (composite) for each star category, I added a normalized score at each possible level (at 4 digits). Applying that to Florida’s current commits and leans (per crystal ball) to get an idea of where they stand provides some insight:
Tyler Booker is practically a 5-star (4.98) and Julian Humphrey is a very high 4-star. This is pretty straightforward. Booker is in the 98th percentile of the 4-star range, so he is a 4.98 star.
Class-wise, Florida has some ground to make up, as they are currently 15th using this metric (and controlling for number of commits/leans). Fortunately they have time:
The responses I saw were interesting. Though I’m not sure if there was a poll or anything statistically done to measure the sentiments, it appeared in the narratives that I saw that more Gator fans would agree to switch coaches than Dawg fans. So, I decided to look into it a little.
Assuming that, as Saban stated, offense is more important than defense my initial hunch was that it may be unwise to switch an offensive wiz like Mullen for a defensive wiz like Smart. Though I think most people that consider Smart to be “better”, think so because of his stellar recruiting levels versus Mullen’s, which is significantly behind teams that are considered elite (Alabama, Clemson, Ohio State, etc.). Smart out-recruits pretty much everyone except Alabama, but Georgia isn’t necessarily performing at the same elite level. Of course, this depends on elite, but ultimately, if you haven’t won a Natty since 1980, I’m not sure how you can make a claim for elite status regardless of recruiting.
Another fact that has me doubting Smart is that he appears to have already peaked. He has the same winning percentage over the last 28 games at ~78% as he did over his first 38 games where Mullen currently sits (at UF). Mullen is currently at 76%, dropping from 82% going into the end of 2020 season where UF dropped 3-straight.
Smart hasn’t improved UGA beyond the initial improvement you often see with a new coach, as his win rate (as above) and even margin of victory isn’t changed much (15 vs 13).
Graph: Smart over time. The below graph shows Smart’s margin of victories over time. The vertical red line is where Smart was at at his 38th game, where Mullen is now at UF. Smart has peaked and then leveled off with a slight decline. While I don’t think his performance is in decline, there is no evidence that it is on the upward swing either.
Smart has done a decent job of performing up to expectations relative to his talent when compared to the rest of the SEC, but he hasn’t exactly exceeded to a great level. In all fairness, his roster puts his expectation levels up there with Alabama, so there’s that. While he may have a comparable roster to Alabama, he certainly doesn’t have the results, and neither does anyone else.
Scatter plot: The below scatterplot depicts performance relative to roster talent. All three seasons under Mullen are above expectations, while Smart has been at expectations twice, above once and below once. It should be noted that it is much harder for Smart to exceed expected win % for his roster talent. This is not so much a knock against him as it is evidence that Mullen does get the most out of his talent. If only he had more, but I digress.
Mullen, however, has outperformed his expected roster talent level each of his 3 years. This says two things: 1) the roster talent level needs to get better and 2) Mullen is good at what we all suspect he is good at, game day and development.
Given that Mullen doesn’t recruit like Smart, it is safe to conclude that he has a lower floor. His loss to LSU last year is an example of that. Typically, Smart can simply out-talent his weaker opponents with a much larger margin for error (ahem, South Carolina?). But offense is key, and points scored are the most straightforward measure of offensive capability. This is why I suspect (I don’t know yet) that Mullen has the higher ceiling. Mullen is more likely to score more points across each category in 5 point increments than Smart in the range of 20 points all the way to 60 points. Every single category. Of note, Smart has the only game between the two with over 65 points, but also has 28 more games at UGA than Mullen has at UF.
Points probability bar graph. The below graph depicts the probability of scoring points for each coach. Mullen scores more points.
The Saban Problem
Regardless of all the intangibles, the most likely scenario for Smart or Mullen to win the SEC and/or a Natty, they are going to have to beat Saban. Since 2006, no SEC team has won a national championship without beating Alabama. As a matter of fact, only one team (2013 FSU) other than Alabama itself has won a national championship without playing Alabama. Furthermore, the only time Alabama was unranked during this current stretch was when the 2006 Florida Gators beat them 28-13, and that was prior to Saban. The average number of points scored by Bama during this run is 37.
In order to beat Saban, you are going to have to score a lot of points. He simply doesn’t lose very many low-scoring games.
Points Against
Saban Win
Opp Win
Saban Win%
< 20
134
3
97.8%
21-30
26
8
76.5%
31-40
5
5
50.0%
41-50
5
7
41.7%
Win/Loss Record vs Nick Saban by Points Allowed
The chance of beating Saban if you score more than 30 points is 54%. Anything less than that and you are at very long odds. The probability of score 30 points in a game is 68% for Mullen (at UF) and 52% for Smart. Smart has failed in 3 tries to reach the 30 point plateau vs Alabama, while Mullen has succeeded in his only shot at Saban. Alabama won all 4 of those games, so it is far from a given that scoring 30 points will beat Saban. But it is your best shot, and Mullen is more likely to reach that level of production, both in general and against Saban.
If Saban is right, and offense is indeed more important, I would take Mullen over Smart. Not because Mullen is guaranteed to necessarily do better, but he is more likely to score more points. Kirby doesn’t seem to have improved as a coach and Mullen doesn’t seem to have improved as a recruiter. However, of the two propositions, it is more likely at this point that Mullen can become a better recruiter more so than Smart can become a better coach. Therefore, I’d go with Mullen. Points and potential over the opposite.
In a nutshell, for either Florida or Georgia to win a championship (SEC or Natty) they will most likely have to beat Alabama. To realistically have a chance to beat Alabama, you have to score 30 or more points. Mullen is more likely to score 30 or more points than Smart. Therefore, Mullen is the choice.
As far as plateauing, it appears Smart already has. We don’t know about Mullen yet, so he may still be ascendant. Again, advantage Mullen.
This is not to say that Mullen is a better overall coach nor that he is guaranteed to perform better. Only that so far, he has shown more potential on offense and offense is needed to overcome the Saban problem. Smart’s recruiting always keeps Georgia in the conversation whereas Mullen can fall off with a bad year since he does not have elite talent. If Georgia gets decent QB play they can potentially win a Natty at any time. If Mullen experiences a rash of injuries, he doesn’t have the depth to overcome that, so this is a fragile situation for him. He needs Emory Jones or AR to be that guy this year to get Florida to the playoffs.
UPDATED on09/19/21. UF lost to Alabama 31-29. Mullen failed to score 30 (on a missed PAT), which would have lead to a tied game (they would miss a 2-point try at the end of the game). That 31st point would have been crucial, as the game would likely have gone to OT which would have favored Florida, as they were grinding Alabama down.
After the game, Coach Mullen reiterated was has been stated here: “… I want a team that can complete for championships. To do that you’re going to have to beat Alabama.”
UPDATED 10/3/2021. Mullen continues his disturbing trend of losing to lesser teams (the floor strikes again). UF lost at Kentucky, 20-13. Meanwhile Georgia demolished 8th ranked Arkansas 37-0. Kirby has a powerful looking team, so he will have his chance to get that elusive championship this year. Smart definitely has an opportunity in front of him, though Alabama still looms large but UF seems to have regressed offensively and may be in for a rough year going forward.
UPDATED 10/16/2021. Mullen continues to nose dive with a loss to unranked LSU while Smart continues to thrive as Georgia throttled Kentucky. Mullen’s future at Florida is being called into question and Smart is cruising toward a Natty. With Alabama seemingly not as good as they normally are, there is only Ohio State who might even pose a threat. Right now the Bulldogs are the clear number one team in the nation.
Georgia and Florida are headed in polar opposite directions, which was highly unlikely. Mullen, after starting 3-0 in one-score games is now 0-7 since. Over the last 10 games to date, Georgia has scored over 30 points in 8 games, averaging over 38 points per game while Florida has surpassed 30 points 7 times and averages 34 points a game. The Bulldogs have shuffled QBs a little, but the unexpected performer there, Stetson Bennett, is playing very well. Contrast that with the fact that UF QB Emory Jones has struggled with interceptions and the team has not played as well with Jones as they do with backup Anthony Richardson.
The two things that were really Dan Mullen’s ace cards when compared to Smart’s, scoring offense and QB play, have both shifted away and now favor Smart.
Clear advantage: Smart.
11/7/2021: Since Florida’s loss to LSU, Mullen has been blown out twice by Georgia and South Carolina. The dramatic disintegration of the program is unexpected but also unacceptable. Mullen, considered a rising star in the coaching profession before this year, is now in danger of losing his job at UF. Wild.
Tombstone
The amazing and precipitous fall of Mullen continued. He lost to a terrible South Carolina team while also overseeing an embarrassing home win versus pathetic Samford, where Samford scored 42 points in the first half and the Gators had to rally to win 70-52. A listless overtime loss to an abysmal Missouri team resulted in Mullen being fired by the Gators. Absolutely stunning turn of events that nobody, certainly not me, saw coming. Meanwhile, Smart is humming along at UGA, and has them still undefeated, ranked number one, and on track to meet Alabama in the SECCG.
Here is a visual on the fall of Mullen and rise of Smart this year:
The green vertical line represents the start of the 2021 season
Mullen has a lot of points below the zero line (losses) and it is clear how this pulls his trend down.
In stark contrast, we can see how this year pulled Smart’s trend way up, something that was not apparent going into this season.
The 30 point barrier
Mullen had been clearly the better coach at scoring point prior to this year. He still owns an overall advantage in scoring 30 points or more (65% v 58%). However, in the 2021 season, Smart is dominating 82% to Mullen’s 55%.
With Mullen fired, this debate is over. What once looked like a promising future for Mullen and Florida has crashed and burned while just the opposite happened at UGA. Good luck to Coach Mullen and his future and THWG.
IN CLOSING (1/11/21):
Mullen failed to develop his highly-rated blue chip QB and couldn’t produce on offense and got fired despite going into the year as a top 5 coach, while Kirby Smart played a walk-on QB over a 5-star and won a natty against Alabama (who had a Heisman-winning QB). Two things with a miniscule probability happened. Amazing
Smart finally scored more than 30 against Saban and picked up that coveted win. Never change, college football. Greatest sport on Earth. Now time for Billy Napier to get to work and get UF back in the mix.
As Nick Saban recently stated, offense is what is predominantly winning games in college football. Furthermore, passing and points are at a premium. Converting on a down and distance (D&D) situation is intuitively a good thing for the offense, which is then good thing for the team’s chances of winning. I was curious about how converting D&D relates to winning. Since I already had the play by play data from the SEC 2020 season courtesy of https://www.collegefootballdata.com/ (with the exception of the Ole Miss Vanderbilt game), I decided to check it out.
Among the 67 regular season games analyzed, the winner of the game converted a D&D situation (either a first down or a touchdown) at a higher rate than their opponent 77.6% of the time. The winner, on average, converted D&D 9.7% more than their opponent. In the few games in which the team with the lower D&D won, this difference shrunk to 4.2%. So in instances when the team with the higher D&D lost, it was typically much closer, which makes sense.
A regression model for the data indicates a pretty strong relationship between winning and D&D conversion success. A statistically significant model (p < .001, r2 = 0.6281). Yea, it’s a small sample size, but still informative. A quick look at a scatterplot shows the linearity:
8 of 14 observations land within 2 standard errors of the regression line. Adding the team labels shows us who underperformed/overperformed as well:
The team that overperformed best was Texas A&M, while Ole Miss underperformed. There were only two games in which the loser of the game had a D&D conversion rate of more than 10% better than their opponent. Mississippi State vs Vanderbilt (MSU won 24-17, but Vanderbilt had a 34.1% D&D while MSU had only a 22.8%) and LSU vs Florida (LSU won, but UF had a D&D of 40.5% compared to 29.1% for LSU). So, going by this, LSU beating Florida was the SEC’s biggest upset in 2020.
Team
avg D&D
win percentage
D&D Rank
Alabama
0.424
1.00
1
Florida
0.390
0.80
2
Ole Miss
0.370
0.44
3
Texas A&M
0.350
0.89
4
Georgia
0.324
0.78
5
Auburn
0.317
0.60
6
LSU
0.315
0.50
7
Missouri
0.308
0.50
8
Arkansas
0.308
0.30
9
Tennessee
0.305
0.30
10
South Carolina
0.301
0.20
11
Kentucky
0.289
0.40
12
Vanderbilt
0.278
0.00
13
Mississippi State
0.275
0.30
14
I’ll probably play with this some more with additional years and conferences. I’d like to see how it plays out with larger sample sizes.
Projecting the Gators’ QB’s total yards and total touchdowns over a 12 game schedule shows just how far ahead of the competition Trask was this year.
So much has already been said about Trask and the fact that his stats came against an all SEC schedule, etc. It seems as if there is some sentiment that Florida’s 3 unfortunate losses are being held against Trask, which is absurd.
The University of Florida Gators 2020 Football Team is highly likely one of the better teams in the nation despite having 3 losses prior to the Cotton Bowl against the University of Oklahoma Sooners. Three low probability plays lead to three narrow defeats, any of which-if prevented- could have resulted in wins instead of losses. If Malik Davis doesn’t fumble on the Gators’ final drive against Texas A&M. If Marco Wilson doesn’t throw a shoe against LSU. If Trey Dean looks left and protects himself against getting vaporized on an interception return against Alabama. Though none of these plays guaranteed victory in any of those games, they were each pivotal in the outcome. They were also each very unlikely to occur, yet they did.
When put into context, the sum of points in those three defeats, is equal to the closest point differential in any of Florida’s 8 victories, 12 against the University of Tennessee. Florida was dominant in all of its games except in its three losses, and in each of those, Florida could have just as easily won, but lady luck was simply not on their side.
The above scatter plot shows data for total yards and touchdowns (rushing and passing) for 103 college QBs in 2020 (https://www.sports-reference.com/cfb/years/2020-passing.html) as they would project over 12 games. Trask projects for the most TDs overall while Ole Miss’ Matt Corral projected to have the most yards, but a big chunk of those yards actually projected from rushing (625). Trask projected to have the most passing yards (4500) and most TDs (47)
Here is another look at the same data with each QB listed:
Touchdown passes by game for SEC QBs that started each gameCumulative yards by passing by SEC QBs who started each game
Since 2005, it appears as if a few teams have become recruiting super powers. Of course, there were good recruiters and power house teams before that. But it seems as if many of the top recruits have been ending up at the same ‘ol schools – Alabama, Ohio State, Clemson, and Georgia. In looking at this, some noticeable trends emerged.
I took all the teams that had an average composite recruit rating of at least 1 standard deviation above the Power 5 mean and separated them out from the rest of college football. The full list is at the bottom of the page. There were 14 teams. I then counted up how many recruits each of these teams had each year back to 2005 that were rated at least 0.9300. It is this rating that I have found to be a sort of ‘cutoff’ to probability for getting drafted by an NFL team. Players rated over 0.9300 are more likely to be drafted than those under.
Below we can see the proportion of how these teams powered up.
So, over time we can see how the concentration of recruiting power has moved from the bottom left to the top right. We can also see how good UF was under Meyer and how bad we were under Mac. Furthermore, it shows that LSU, Alabama, and Georgia are soaking up a ton of talent in the SEC, making the task of winning the conference (if you’re not one of those teams) probably much harder.
The data:
Team
overall avg
rank
Standardized
5-stars
4-stars
3-stars
Conf.
USC
0.9214
1
2.177
48
154
99
Pac-12
Ohio State
0.9201
2
2.133
32
188
82
Big Ten
Alabama
0.9169
3
2.021
52
202
126
SEC
Texas
0.9126
4
1.866
24
204
116
Big 12
Georgia
0.9102
5
1.783
42
174
135
SEC
Florida
0.9091
6
1.744
30
177
132
SEC
LSU
0.9081
7
1.710
27
192
133
SEC
Florida State
0.9036
8
1.549
36
152
153
ACC
Notre Dame
0.9023
9
1.504
13
180
142
Ind
Oklahoma
0.8982
10
1.359
15
162
152
Big 12
Michigan
0.8939
11
1.208
12
173
167
Big Ten
Clemson
0.8895
12
1.052
25
125
168
ACC
Miami
0.8894
13
1.049
13
131
175
ACC
Auburn
0.8886
14
1.021
13
155
178
SEC
The above graph shows the recruiting classes for each SEC team from 2005-2020The above graph shows each SEC team’s recruiting as it correlates with with percentage
The Fair Catch 