Cutler or McCown? The NFL’s Sample Size Problem

JayJoshHug
Photo Credit: Chicago Bears Official Site

There are 16 games in an NFL season. That is not breaking news for you, I’m sure. But it creates a problem when evaluating player performances; as anyone with a passing interest in sabermetrics can tell you, 16 games for a baseball player is not nearly enough to tell you anything statistically significant. But for football, that’s what we’re stuck with. Obviously the two sports are not strictly analogous; a quarterback has an active role in many more plays in any given game than any single baseball player would. So in that sense, we can mine more data on a per-game basis than we can for a baseball player. Which has some merit, for sure; I’m not saying that there are zero valuable statistics in football. I think there are multiple metrics that can be used to discern how well a player performed in any given game, or over the course of the season.

But I think the problem comes into play when people attempt to use stats derived from an inherently small sample to predict player performance. This phenomenon is currently playing a major role in the ongoing media debate surrounding the potential return of Jay Cutler. By any reasonable metric, Josh McCown has been very good, and that includes the “eye test.” By all the same measures, Jay Cutler was very good as well, when healthy. But due to a variety of factors (the obvious coolness of McCown’s story and the general dislike of Cutler (itself a tired and by now obsolete narrative) chief among them) there has been a growing chorus of voices who think McCown should remain the starter regardless of Cutler’s health situation.

I can understand that urge. No one ever wants to rock the boat, and the Bears have waited so long for a competent offense that it’s easy to want to handle it gingerly, like a baby bird, afraid to break something so seemingly fragile. But when you consider that the offense didn’t miss a beat with the transition to McCown, I think it’s fair to speculate that they wouldn’t miss a beat with the transition back to Cutler, and if any bumps were to arise I doubt they’d be too jarring. Which leaves the “McCown is just better” crowd, and those people have to be basing their case strictly on McCown’s performance this season; before this year, he had done little that would lead you to believe he was capable of playing as well as he’s played this season. And make no mistake about it, he’s playing well. His performance Monday night tied for the second-highest rated quarterback performance of the season according to ESPN’s QBR metric. (Which isn’t a perfect stat, but it’s better than passer rating, in my opinion. I’ll also note that if a Cowboy catches either of the two balls McCown threw right to defenders, or if the refs don’t throw a flag for defensive holding on a ball that was actually intercepted, things might look a little different.) But here we run into the problem with the sample size, as McCown has started just five games. Jay Cutler has started and finished six games, and he only played poorly in one of them, the Week 4 loss to Detroit. McCown’s body of work is more recent, and his performance Monday was the best performance by either quarterback this season, but it also came against the league’s worst passing defense. Context matters.

McCown’s performance has also been buoyed by the very impressive play of his skill position players, notably the suddenly crazy exploits of Alshon Jeffery. Marc Trestman’s game-planning, play-calling, and game-management have all been very, very good. Those things would be there for Jay Cutler, as well; McCown is not doing things that Cutler is incapable of doing. (Well, maybe the spinning, Elway-like dive into the end zone; I’m not sure Cutler’s ankle would be up for that one.) But because he’s played nearly a half-season’s worth of games, it becomes easy for fans, media, and players alike to view that as a large enough sample to justify an ongoing entrenchment as the starter.

But that’s not enough. Data collected from 5.5 games just isn’t enough to be used to gauge future performance. On Twitter, I equated this “QB controversy” to an above-average MLB player missing two weeks, while his below-average replacement filled in and played well. That drew questions as to the validity of that analogy, and I’d like to try to clear that up a bit. As was noted, 5 games in the NFL roughly correlates to 50 MLB games in terms of share of the season. But it does not correlate to 50 games in terms of statistical sampling. (I’ll also note that 50 games is still a small sample by baseball standards.) It’s not a difficult difference to illustrate; football teams within the past decade have finished the regular season with every possible finishing record, from 16-0 to 0-16. Teams in the NFL routinely go on lengthy winning streaks; the Chiefs began this season at 9-0. But no MLB team has ever gone 162-0, or 0-162. No team wins or loses 90 games in a row. The NFL is often praised for its parity, the idea that any team could beat any team at any time, or that any team could make the playoffs in any given year.

Those things are true, but the first point is true in baseball as well. The second part isn’t, because the sample size is much greater. If the MLB season were 16 games, any team could make the playoffs. (I’m using team results to compare the leagues, but the sample size issues work for analyzing individuals as well.) The sports are different, and it’s not an apples-to-apples comparison (the nature of baseball success is more random, which certainly plays into this discussion, although not to a degree that makes me think my greater point is incorrect.) But there’s no way that five games can tell us anything useful about McCown’s likelihood of sustaining success, no more than Rex Grossman’s various multi-game hot streaks told us about his ability to be consistently successful.

I think the fact that Marc Trestman has been unwavering in his support of Cutler should give you a good idea as to who should start. If you wanted to read into it further, you could probably look at it as insight into their organizational belief in Cutler. This is a team that is fighting for a playoff spot, coached by one of the more analytically minded men in the league, who works for one of the more analytically minded general managers. Neither of them was involved with acquiring Cutler (who will be a free agent after this year), and they know how well McCown has played. If there was ever a time for the Bears to make a move similar to San Francisco’s benching of Alex Smith from last season, this would be it. But all indications are that if Cutler can play, he’ll play. That should tell you more about the gulf between their abilities than anything I can say, and certainly more than anything six games worth of stats could tell you.

It’s also important to note that if Jay comes back and doesn’t play well, that doesn’t mean they made the wrong call. The results might not have been what they wanted, but if they believe that a healthy Cutler gives them a better chance to win than McCown would give them, they have to do it. Cutler not playing well wouldn’t automatically mean that McCown would have been better. But let’s hope we won’t have to worry about that particular narrative twist.

Anyway, it’s a good problem to have, isn’t it? Beats the hell out of debating Grossman vs. Orton.

written by

Brett Taylor is the Lead Cubs Writer at Bleacher Nation, and you can find him on Twitter at @BleacherNation and on LinkedIn here. Brett is also the founder of Bleacher Nation, which opened up shop in 2008 as an independent blog about the Chicago Cubs. Later growing to incorporate coverage of other Chicago sports, Bleacher Nation is now one of the largest regional sports blogs on the web.

more bears news

Rome Odunze Has a Visit With the Bears Coming Soon

If you aren't already familiar with Rome Odunze's name or game, your time will come. The University of Washington wide receiver is one of the top prospects in the 2024 NFL Draft. At this point of the year, I've officially...

Matt Eberflus Opens a Window to the Chicago Bears Draft Plans

NFL coaches are notoriously tight-lipped. And in his first two seasons as Bears head coach, Matt Eberflus has been no different. But every once in a while you'll catch Eberflus peel back the curtain and provide some insight regarding the...

What Does a Chicago Bears NFL Draft Trade Down Look Like?

You'd think we'd be well-versed in trade-back options after months of wondering what kind of haul it would take for the Bears to move off taking quarterback Caleb Williams with the first pick of the 2024 NFL Draft. But trading...

D-Line Thoughts, Caleb, GM Seth Rollins, Pack Votes No, and Other Bears Bullets

Cubs and White Sox opening day is today today. March Madness resumes with the Fighting Illini playing in the Sweet 16 later tonight. SIU, my alma mater, is reportedly hiring a "Coach Nagy" to lead its basketball team. And what...

Dan Campbell is Happy To See Justin Fields Leave the NFC North

There was no escaping the mixed emotions from Chicago Bears fans when the Justin Fields trade went down earlier this month. But there were none of that from Detroit Lions Head Coach Dan Campbell. Heck, I'd argue there was just...

Another Pass Rusher Signs Elsewhere: Jadeveon Clowney Lands With the Panthers

Another day. Another pass rusher signing. And another defensive lineman going to a team that isn't the Chicago Bears. NFL Network insider Ian Rapoport was the first with the news of Jadeveon Clowney signing with the Carolina Panthers. Meanwhile, ESPN's...

Malik Nabers Might’ve Run His Way Out of the Chicago Bears Draft Spot

Today was LSU's Pro Day and Chicago Bears GM Ryan Poles was scheduled to be in attendance, per The Athletic's Adam Jahns. Assuming Poles made his flight, he got a good look at the likes of quarterback Jayden Daniels, as...

Goodell on Bears Stadium Deal, No to Hard Knocks? More Old Friends Go to Steelers, and Other Bears Bullets

I keep forgetting that I have a Starbucks gift card from Christmas that has gone unused to this point. Maybe I'll remember it next time I go out for coffee. A big shout out to PFF Brad, who made the...

Ryan Poles Was Pissed Off By RGIII’s Comments About Caleb Williams Situation

When I saw them pop up, RGIII's comments suggesting Caleb Williams pull an Eli Manning and refuse to play for the Chicago Bears were met with an eye-roll, audible groan, and immediate dismissal before moving onto more pertinent topics. But...

Bears Stadium Update: Kevin Warren Reiterates Plans to Build in Downtown Chicago

It sure is starting to sound like Arlington Heights fumbled the bag when it comes to locking in Arlington Park as the future home of a Chicago Bears stadium. While meeting with the media at the NFL owners meetings in...

Latest News

Diamondbacks vs. Rockies: Start Time, Streaming Live, TV Channel, How to Watch

In their season opener on Friday, Corbin Carroll and the Arizona Diamondbacks take on Nolan Jones and the Colorado Rockies, with first pitch scheduled for 9:40 PM ET at Chase Field.Keep reading to find out how to watch the Arizona-Colorado...

Guardians vs. Athletics: Start Time, Streaming Live, TV Channel, How to Watch

Logan Allen will take the mound for the Cleveland Guardians when they square off against starter Ross Stripling and the Oakland Athletics in their season opener on Friday, with first pitch at 9:40 PM ET.Find out how to watch the...

How to Watch Mariners vs. Red Sox: Live Stream or on TV

In their season opener on Friday, Julio Rodriguez and the Seattle Mariners play Rafael Devers and the Boston Red Sox, with first pitch scheduled for 9:40 PM ET at T-Mobile Park.Keep reading to see how to watch the Seattle-Boston matchup...

How to Watch Padres vs. Giants: Live Stream or on TV

Fernando Tatis Jr. and the San Diego Padres (2-1) host Jorge Soler and the San Francisco Giants (0-1) on Friday, with first pitch at 9:40 PM ET.The article below will provide you with everything you need to know to watch...

Dodgers vs. Cardinals: Start Time, Streaming Live, TV Channel, How to Watch

The St. Louis Cardinals (0-1), led by Paul Goldschmidt, visit Mookie Betts and the Los Angeles Dodgers (2-1) at 10:10 PM ET on Friday.Keep scrolling to find out all the details on how to watch Los Angeles play St. Louis...

OMG: Michael Busch Scored the 9th Inning, GO-AHEAD Run, Because He Just Never Stopped Running

That was absolutely incredible. To all the little kids out there, when your coach tells you to never stop running, this is why. Rookie Cubs 1B Michael Busch (the guy the Cubs got from the Dodgers this offseason to be...

Astros vs. Yankees: Start Time, Streaming Live, TV Channel, How to Watch

The Houston Astros (0-1), with Cristian Javier on the mound, will host the New York Yankees (1-0), who will send out Carlos Rodon, at 8:10 PM ET on Friday.If you're trying to watch this game, keep reading for streaming and...

Rays vs. Blue Jays: Free Live Stream, TV Channel, How to Watch

The Toronto Blue Jays (1-0), led by Vladimir Guerrero Jr., visit Yandy Diaz and the Tampa Bay Rays (0-1) at 6:50 PM ET on Friday.Find out how to watch the Tampa Bay-Toronto matchup below.When is Rays vs. Blue Jays and...

Enhanced Box Score: Rangers 4, Cubs 3 – March 28, 2024

It took exactly one day for the Cubs' ace to sustain what may be a significant injury, for their closer to blow a save, for the umpires to significantly impact the outcome of a critical play (this time in the...

Justin Steele Injury Update: New Classification, MRI Tomorrow, IL Stint Coming? Steele and Counsell Speak

A significant update on Justin Steele came from Craig Counsell tonight after the Cubs extra innings loss in Texas. In short: Yes, a stint on the Injured List is likely for Justin Steele. But we still don't know how severe...

more bears news