Cutler or McCown? The NFL’s Sample Size Problem

JayJoshHug
Photo Credit: Chicago Bears Official Site

There are 16 games in an NFL season. That is not breaking news for you, I’m sure. But it creates a problem when evaluating player performances; as anyone with a passing interest in sabermetrics can tell you, 16 games for a baseball player is not nearly enough to tell you anything statistically significant. But for football, that’s what we’re stuck with. Obviously the two sports are not strictly analogous; a quarterback has an active role in many more plays in any given game than any single baseball player would. So in that sense, we can mine more data on a per-game basis than we can for a baseball player. Which has some merit, for sure; I’m not saying that there are zero valuable statistics in football. I think there are multiple metrics that can be used to discern how well a player performed in any given game, or over the course of the season.

But I think the problem comes into play when people attempt to use stats derived from an inherently small sample to predict player performance. This phenomenon is currently playing a major role in the ongoing media debate surrounding the potential return of Jay Cutler. By any reasonable metric, Josh McCown has been very good, and that includes the “eye test.” By all the same measures, Jay Cutler was very good as well, when healthy. But due to a variety of factors (the obvious coolness of McCown’s story and the general dislike of Cutler (itself a tired and by now obsolete narrative) chief among them) there has been a growing chorus of voices who think McCown should remain the starter regardless of Cutler’s health situation.

I can understand that urge. No one ever wants to rock the boat, and the Bears have waited so long for a competent offense that it’s easy to want to handle it gingerly, like a baby bird, afraid to break something so seemingly fragile. But when you consider that the offense didn’t miss a beat with the transition to McCown, I think it’s fair to speculate that they wouldn’t miss a beat with the transition back to Cutler, and if any bumps were to arise I doubt they’d be too jarring. Which leaves the “McCown is just better” crowd, and those people have to be basing their case strictly on McCown’s performance this season; before this year, he had done little that would lead you to believe he was capable of playing as well as he’s played this season. And make no mistake about it, he’s playing well. His performance Monday night tied for the second-highest rated quarterback performance of the season according to ESPN’s QBR metric. (Which isn’t a perfect stat, but it’s better than passer rating, in my opinion. I’ll also note that if a Cowboy catches either of the two balls McCown threw right to defenders, or if the refs don’t throw a flag for defensive holding on a ball that was actually intercepted, things might look a little different.) But here we run into the problem with the sample size, as McCown has started just five games. Jay Cutler has started and finished six games, and he only played poorly in one of them, the Week 4 loss to Detroit. McCown’s body of work is more recent, and his performance Monday was the best performance by either quarterback this season, but it also came against the league’s worst passing defense. Context matters.

McCown’s performance has also been buoyed by the very impressive play of his skill position players, notably the suddenly crazy exploits of Alshon Jeffery. Marc Trestman’s game-planning, play-calling, and game-management have all been very, very good. Those things would be there for Jay Cutler, as well; McCown is not doing things that Cutler is incapable of doing. (Well, maybe the spinning, Elway-like dive into the end zone; I’m not sure Cutler’s ankle would be up for that one.) But because he’s played nearly a half-season’s worth of games, it becomes easy for fans, media, and players alike to view that as a large enough sample to justify an ongoing entrenchment as the starter.

But that’s not enough. Data collected from 5.5 games just isn’t enough to be used to gauge future performance. On Twitter, I equated this “QB controversy” to an above-average MLB player missing two weeks, while his below-average replacement filled in and played well. That drew questions as to the validity of that analogy, and I’d like to try to clear that up a bit. As was noted, 5 games in the NFL roughly correlates to 50 MLB games in terms of share of the season. But it does not correlate to 50 games in terms of statistical sampling. (I’ll also note that 50 games is still a small sample by baseball standards.) It’s not a difficult difference to illustrate; football teams within the past decade have finished the regular season with every possible finishing record, from 16-0 to 0-16. Teams in the NFL routinely go on lengthy winning streaks; the Chiefs began this season at 9-0. But no MLB team has ever gone 162-0, or 0-162. No team wins or loses 90 games in a row. The NFL is often praised for its parity, the idea that any team could beat any team at any time, or that any team could make the playoffs in any given year.

Those things are true, but the first point is true in baseball as well. The second part isn’t, because the sample size is much greater. If the MLB season were 16 games, any team could make the playoffs. (I’m using team results to compare the leagues, but the sample size issues work for analyzing individuals as well.) The sports are different, and it’s not an apples-to-apples comparison (the nature of baseball success is more random, which certainly plays into this discussion, although not to a degree that makes me think my greater point is incorrect.) But there’s no way that five games can tell us anything useful about McCown’s likelihood of sustaining success, no more than Rex Grossman’s various multi-game hot streaks told us about his ability to be consistently successful.

I think the fact that Marc Trestman has been unwavering in his support of Cutler should give you a good idea as to who should start. If you wanted to read into it further, you could probably look at it as insight into their organizational belief in Cutler. This is a team that is fighting for a playoff spot, coached by one of the more analytically minded men in the league, who works for one of the more analytically minded general managers. Neither of them was involved with acquiring Cutler (who will be a free agent after this year), and they know how well McCown has played. If there was ever a time for the Bears to make a move similar to San Francisco’s benching of Alex Smith from last season, this would be it. But all indications are that if Cutler can play, he’ll play. That should tell you more about the gulf between their abilities than anything I can say, and certainly more than anything six games worth of stats could tell you.

It’s also important to note that if Jay comes back and doesn’t play well, that doesn’t mean they made the wrong call. The results might not have been what they wanted, but if they believe that a healthy Cutler gives them a better chance to win than McCown would give them, they have to do it. Cutler not playing well wouldn’t automatically mean that McCown would have been better. But let’s hope we won’t have to worry about that particular narrative twist.

Anyway, it’s a good problem to have, isn’t it? Beats the hell out of debating Grossman vs. Orton.

written by

Brett Taylor is the Lead Cubs Writer at Bleacher Nation, and you can find him on Twitter at @BleacherNation and on LinkedIn here. Brett is also the founder of Bleacher Nation, which opened up shop in 2008 as an independent blog about the Chicago Cubs. Later growing to incorporate coverage of other Chicago sports, Bleacher Nation is now one of the largest regional sports blogs on the web.

more bears news

D-Line Thoughts, Caleb, GM Seth Rollins, Pack Votes No, and Other Bears Bullets

Cubs and White Sox opening day is today today. March Madness resumes with the Fighting Illini playing in the Sweet 16 later tonight. SIU, my alma mater, is reportedly hiring a "Coach Nagy" to lead its basketball team. And what...

Dan Campbell is Happy To See Justin Fields Leave the NFC North

There was no escaping the mixed emotions from Chicago Bears fans when the Justin Fields trade went down earlier this month. But there were none of that from Detroit Lions Head Coach Dan Campbell. Heck, I'd argue there was just...

Another Pass Rusher Signs Elsewhere: Jadeveon Clowney Lands With the Panthers

Another day. Another pass rusher signing. And another defensive lineman going to a team that isn't the Chicago Bears. NFL Network insider Ian Rapoport was the first with the news of Jadeveon Clowney signing with the Carolina Panthers. Meanwhile, ESPN's...

Malik Nabers Might’ve Run His Way Out of the Chicago Bears Draft Spot

Today was LSU's Pro Day and Chicago Bears GM Ryan Poles was scheduled to be in attendance, per The Athletic's Adam Jahns. Assuming Poles made his flight, he got a good look at the likes of quarterback Jayden Daniels, as...

Goodell on Bears Stadium Deal, No to Hard Knocks? More Old Friends Go to Steelers, and Other Bears Bullets

I keep forgetting that I have a Starbucks gift card from Christmas that has gone unused to this point. Maybe I'll remember it next time I go out for coffee. A big shout out to PFF Brad, who made the...

Ryan Poles Was Pissed Off By RGIII’s Comments About Caleb Williams Situation

When I saw them pop up, RGIII's comments suggesting Caleb Williams pull an Eli Manning and refuse to play for the Chicago Bears were met with an eye-roll, audible groan, and immediate dismissal before moving onto more pertinent topics. But...

Bears Stadium Update: Kevin Warren Reiterates Plans to Build in Downtown Chicago

It sure is starting to sound like Arlington Heights fumbled the bag when it comes to locking in Arlington Park as the future home of a Chicago Bears stadium. While meeting with the media at the NFL owners meetings in...

The NFL is Playing a Christmas Day Doubleheader in 2024

Christmastime is generally full of cheer. And in 2024, it will be full of cheering. In fact, we're talking double the cheer. ESPN NFL insider Adam Schefter reports the league will play not one, but *TWO* Christmas Day games. It's...

2024 NFL Mock Draft: How High Can J.J. McCarthy’s Stock Rise?

Hard to believe we're less than a month away from the 1st Round of the 2024 NFL Draft! With just weeks remaining until the NFL's most noteworthy offseason event, there are plenty of rumors floating. As we work through all...

NFL Rules Shakeup, Tomlin on Fields, Trubisky’s Linsanity Run, and Other Bears Bullets

The Bachelor's season-finale had a couple of jaw-dropping moments. Once the group chat catches up, it is going to be lit. What better way to start Bears Bullets than with breakfast anecdotes? I think I'm going to enjoy D'Andre Swift's...

Latest News

Buffalo Bills 1st Round Mock Draft: Wild Crossroads at 28 – Offense or Defense?

The Buffalo Bills stand at a crossroads. The 2024 NFL Draft looms, and the 28th pick presents a fascinating dilemma. This Buffalo Bills 1st Round Mock Draft analyzes 10 different options that could be available to them at their current...

How to Watch Celtics vs. Hawks: Live Stream or on TV

The Atlanta Hawks (33-39) hope to continue a three-game home winning stretch when hosting the Boston Celtics (57-15) on Thursday, March 28, 2024 at 7:30 PM ET.In their last time out, the Celtics lost 120-118 to the Hawks on Monday....

Rangers vs. Cubs Prediction: Expert Picks, Odds, Stats & Best Bets – Thursday, March 28, 2024

The Texas Rangers open their season as -120 favorites in a home matchup with the Chicago Cubs on Thursday at 7:35 PM ET, live on ESPN.Nathan Eovaldi will get the ball for Texas while Chicago will counter with Justin Steele.Rangers...

Dejounte Murray Player Props: Three-Pointer Props and Odds vs. the Celtics – Thursday, March 28, 2024

Will Dejounte Murray make four 3-pointers when the Atlanta Hawks face the Boston Celtics on Thursday at 7:30 PM ET? If so, he'll beat his prop for the game.Dejounte Murray's Three-Pointer Prop Odds vs. the CelticsDejounte Murray's over/under for three-pointers...

Bogdan Bogdanovic Player Props: Three-Pointer Props and Odds vs. the Celtics – Thursday, March 28, 2024

To beat his 3-point over/under, Bogdan Bogdanovic needs to make three from beyond the arc on Thursday, when his Atlanta Hawks meet the Boston Celtics on Thursday at 7:30 PM ET.Bogdan Bogdanovic's Three-Pointer Prop Odds vs. the CelticsBogdan Bogdanovic's over/under...

Jayson Tatum Player Props: Three-Pointer Props and Odds vs. the Hawks – Thursday, March 28, 2024

At 7:30 PM ET on Thursday, it'll be the Boston Celtics versus the Atlanta Hawks, with Jayson Tatum needing three treys to beat his 3-pointer prop.Jayson Tatum's Three-Pointer Prop Odds vs. the HawksJayson Tatum's over/under for three-pointers in this game:...

Jaylen Brown Player Props: Three-Pointer Props and Odds vs. the Hawks – Thursday, March 28, 2024

To beat his 3-point prop, Jaylen Brown must make three from beyond the arc on Thursday, when his Boston Celtics play the Atlanta Hawks on Thursday at 7:30 PM ET.Jaylen Brown's Three-Pointer Prop Odds vs. the HawksJaylen Brown's over/under for...

Derrick White Player Props: Three-Pointer Props and Odds vs. the Hawks – Thursday, March 28, 2024

To beat his 3-point prop, Derrick White needs to make three from beyond the arc on Thursday, when his Boston Celtics meet the Atlanta Hawks, starting at 7:30 PM ET.Derrick White's Three-Pointer Prop Odds vs. the HawksDerrick White's over/under for...

BetMGM Promo Code Boosts Your First Bet Up to $1,500 for MLB, NCAA Tourney, and Beyond

The 2024 baseball season is upon us! But before the first pitch is thrown, why not enhance your MLB betting experience with the BetMGM Promo Code welcome offer. With Opening Day right around the corner, baseball fans are eager to...

WOW: Minnesota Timberwolves Sale to Alex Rodriguez Falls Through

The Minnesota Timberwolves dropped a bomb on Thursday morning. According to a statement released by the Timberwolves and Lynx ownership group, the franchise is no longer for sale. A transition of power was thought to be in place over the...

more bears news