A 5-star bias? A case study.

247 sports, a college football recruiting heavyweight, just released their top 25 running backs list for the 2019 college football season. They also included 10 ‘Honorable Mentions’.  Here is the list (Honorable mentions in blue, all tied for 26th rank):

Rank Player Team HS Year HS Rating HS Stars
26 Stevie Scott Indiana 2018 0.8402 3
26 Brian Robinson Alabama 2017 0.9361 4
26 Isaiah Bowser Northwestern 2018 0.8639 3
26 Larry Rountree III Missouri 2017 0.8435 3
26 Kylin Hill  Mississippi State 2017 0.9184 4
26 Jordan Cronkrite USF 2015 0.8853 3
26 Anthony McFarland Maryland 2017 0.9537 4
26 Master Teague Ohio State 2018 0.9132 4
26 Chuba Hubbard Oklahoma State 2017 0.8868 3
26 Trey Sermon Oklahoma   2017 0.9232 4
25 Zack Moss Utah 2016 0.8389 3
24 Spencer Brown UAB 2017 0.758 2
23 Pooka Williams Kansas 2018 0.9055 4
22 Darrynton Evans Appalachian State 2015 0.7519 2
21 Joshua Kelley UCLA 2015 0.7667 2
20 Rakeem Boyd Arkansas 2018 0.8467 3
19 Ben LeMay Charlotte 2016 0.8273 3
18 Lamical Perine Florida 2016 0.8699 3
17 Salvon Ahmed Washington 2017 0.9476 4
16 Ricky Slade Penn State 2018 0.9853 5
15 Cam Akers FSU 2017 0.9984 5
14 CJ Verdell Oregon 2017 0.8752 3
13 Ke’Shawn Vaughn Vanderbilt 2015 0.8953 4
12 Michael Warren II Cincinnati 2017 0.8707 3
11 J.J. Taylor Arizona 2016 0.8396 3
10 Kennedy Brooks Oklahoma 2017 0.9159 4
9 Jermar Jefferson Oregon State 2018 0.8619 3
8 Greg McCrae UCF 2016 0.8135 3
7 Najee Harris Alabama 2017 0.9984 5
6 J.K. Dobbins Ohio State 2017 0.9791 4
5 D’Andre Swift Georgia 2017 0.9838 5
4 AJ Dillon Boston College 2017 0.8803 3
3 Eno Benjamin Arizona State 2017 0.94 4
2 Jonathan Taylor Wisconsin 2017 0.8854 3
1 Travis Etienne Clemson 2017 0.9171 4

There’s nothing on the list that I really care to disagree with. As a Gator fan, I would Perine number one, but there may be some bias occurring there. I certainly think he is better than 18th, but I digress…

5starscores

The 2018 statistics were taken for each player on the list and analyzed. Of note, only the stats for those players who were in the top 290 in performance last year were included (because this is how many were available at my source). In the above table, the production for each of the players on 247’s list is displayed. There were four 5-star players on the list and three 2-stars, comprising 11% and 8 % of the list, respectively. The key statistic here is where each of the star categories averaged their rank on the list. The 5-stars averaged the 11th (10.8) overall ranking and the 2-stars averaged 22nd (22.3).  4-stars were ranked evenly with 3-stars on average, 16.9 to 17.0.

When we take each of these categories and ranking them from 1 to 4 (1 being the highest), here is what we get:

5staravgs

The 5-star running backs averaged the best ranking, number of receptions, and receiving touchdowns. However, they were last in rushing yards, rushing touchdowns, plays from scrimmage, total yards, and total touchdowns. Overall, their production was barely better than that of the 2-stars (2.8 to 2.9) on average across the categories.

For the statistics folks, the difference between the 2-star averages in terms of where they are ranked on the list and that of the 5-star averages is statistically significant:

An independent-samples t-test was conducted to compare the rank of 2-star and 5-star players. Given a violation of Levene’s test for homogeneity of variances, (F= 32.775, p = .002), a t-test not assuming homogeneous variances was calculated. There was a significant difference in the scores for the 2-stars (M=22.33, SD=1.528) and 5-stars (M=10.75, SD=5.560) conditions; t(3.58)= 3.436, p= 0.02. The size of this effect (d = 2.84), as indexed by Cohen’s (1988) coefficient d was found to exceed the convention for a large effect size (d = 0.80).

These results suggest that 5-star players were ranked significantly higher on this list than were 2-star players. But did the production between the two groups warrant the disparity in the ranking?

An independent-samples t-test indicated that total yards were not significantly higher for
5-stars (M = 995.66, SD = 304.93) than for 2-stars (M = 1319.0, SD = 102.22), t(4) = 1.741,
p = .157, d = 0.35. Equal variances were assumed. 

Ricky Slade from Penn State was not included in the statistical comparison of yards because he wasn’t among the top 290 performers last year. His low production scores would’ve drug the 5-stars overall average down. (Did he redshirt last year?).

So why the disparity? 5-star running backs are given the highest ranking, but overall had the lowest production (if you included Slade) of each of the groups? I think there was certainly a bias for those players in effect in this particular ranking. When we look at each player’s high school year and average them out per star ranking, we can see the 5-stars are typically newer. This goes down the line:

Stars Avg HS yr
5 2017.3
4 2017
3 2016.8
2 2015.7

As you can see, the lower the star ranking, the more likely the player was to have been in college longer. This players ranking was likely on potential and subjective opinion.

Disclaimer because some college football fans get upset about everything:

This is just a case study of one list that was put out by 247 sports. Their list may be complete garbage and invalid in every way, not representative of the population, etc. I know. The point here was to take a micro-level look at the potential bias that occurs when sports reports, journalists, etc do these rankings.

Yes, the 5-star players play against tougher competition than the typical 2-star. Yes, that matters. But they don’t play against tougher competition than the 4-stars and most of the 3-stars.

Also: Alabama (Brian Robinson and Najee Harris) and Ohio State (Master Teague and JK Dobbins) have 2 players each on the list. Florida has 2 players it recruited on the list (LaMical Perine and Jordan Cronkite), but Cronkite transferred (now as USF).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s