Cutler or McCown? The NFL’s Sample Size Problem

JayJoshHug
Photo Credit: Chicago Bears Official Site

There are 16 games in an NFL season. That is not breaking news for you, I’m sure. But it creates a problem when evaluating player performances; as anyone with a passing interest in sabermetrics can tell you, 16 games for a baseball player is not nearly enough to tell you anything statistically significant. But for football, that’s what we’re stuck with. Obviously the two sports are not strictly analogous; a quarterback has an active role in many more plays in any given game than any single baseball player would. So in that sense, we can mine more data on a per-game basis than we can for a baseball player. Which has some merit, for sure; I’m not saying that there are zero valuable statistics in football. I think there are multiple metrics that can be used to discern how well a player performed in any given game, or over the course of the season.

But I think the problem comes into play when people attempt to use stats derived from an inherently small sample to predict player performance. This phenomenon is currently playing a major role in the ongoing media debate surrounding the potential return of Jay Cutler. By any reasonable metric, Josh McCown has been very good, and that includes the “eye test.” By all the same measures, Jay Cutler was very good as well, when healthy. But due to a variety of factors (the obvious coolness of McCown’s story and the general dislike of Cutler (itself a tired and by now obsolete narrative) chief among them) there has been a growing chorus of voices who think McCown should remain the starter regardless of Cutler’s health situation.

I can understand that urge. No one ever wants to rock the boat, and the Bears have waited so long for a competent offense that it’s easy to want to handle it gingerly, like a baby bird, afraid to break something so seemingly fragile. But when you consider that the offense didn’t miss a beat with the transition to McCown, I think it’s fair to speculate that they wouldn’t miss a beat with the transition back to Cutler, and if any bumps were to arise I doubt they’d be too jarring. Which leaves the “McCown is just better” crowd, and those people have to be basing their case strictly on McCown’s performance this season; before this year, he had done little that would lead you to believe he was capable of playing as well as he’s played this season. And make no mistake about it, he’s playing well. His performance Monday night tied for the second-highest rated quarterback performance of the season according to ESPN’s QBR metric. (Which isn’t a perfect stat, but it’s better than passer rating, in my opinion. I’ll also note that if a Cowboy catches either of the two balls McCown threw right to defenders, or if the refs don’t throw a flag for defensive holding on a ball that was actually intercepted, things might look a little different.) But here we run into the problem with the sample size, as McCown has started just five games. Jay Cutler has started and finished six games, and he only played poorly in one of them, the Week 4 loss to Detroit. McCown’s body of work is more recent, and his performance Monday was the best performance by either quarterback this season, but it also came against the league’s worst passing defense. Context matters.

McCown’s performance has also been buoyed by the very impressive play of his skill position players, notably the suddenly crazy exploits of Alshon Jeffery. Marc Trestman’s game-planning, play-calling, and game-management have all been very, very good. Those things would be there for Jay Cutler, as well; McCown is not doing things that Cutler is incapable of doing. (Well, maybe the spinning, Elway-like dive into the end zone; I’m not sure Cutler’s ankle would be up for that one.) But because he’s played nearly a half-season’s worth of games, it becomes easy for fans, media, and players alike to view that as a large enough sample to justify an ongoing entrenchment as the starter.

But that’s not enough. Data collected from 5.5 games just isn’t enough to be used to gauge future performance. On Twitter, I equated this “QB controversy” to an above-average MLB player missing two weeks, while his below-average replacement filled in and played well. That drew questions as to the validity of that analogy, and I’d like to try to clear that up a bit. As was noted, 5 games in the NFL roughly correlates to 50 MLB games in terms of share of the season. But it does not correlate to 50 games in terms of statistical sampling. (I’ll also note that 50 games is still a small sample by baseball standards.) It’s not a difficult difference to illustrate; football teams within the past decade have finished the regular season with every possible finishing record, from 16-0 to 0-16. Teams in the NFL routinely go on lengthy winning streaks; the Chiefs began this season at 9-0. But no MLB team has ever gone 162-0, or 0-162. No team wins or loses 90 games in a row. The NFL is often praised for its parity, the idea that any team could beat any team at any time, or that any team could make the playoffs in any given year.

Those things are true, but the first point is true in baseball as well. The second part isn’t, because the sample size is much greater. If the MLB season were 16 games, any team could make the playoffs. (I’m using team results to compare the leagues, but the sample size issues work for analyzing individuals as well.) The sports are different, and it’s not an apples-to-apples comparison (the nature of baseball success is more random, which certainly plays into this discussion, although not to a degree that makes me think my greater point is incorrect.) But there’s no way that five games can tell us anything useful about McCown’s likelihood of sustaining success, no more than Rex Grossman’s various multi-game hot streaks told us about his ability to be consistently successful.

I think the fact that Marc Trestman has been unwavering in his support of Cutler should give you a good idea as to who should start. If you wanted to read into it further, you could probably look at it as insight into their organizational belief in Cutler. This is a team that is fighting for a playoff spot, coached by one of the more analytically minded men in the league, who works for one of the more analytically minded general managers. Neither of them was involved with acquiring Cutler (who will be a free agent after this year), and they know how well McCown has played. If there was ever a time for the Bears to make a move similar to San Francisco’s benching of Alex Smith from last season, this would be it. But all indications are that if Cutler can play, he’ll play. That should tell you more about the gulf between their abilities than anything I can say, and certainly more than anything six games worth of stats could tell you.

It’s also important to note that if Jay comes back and doesn’t play well, that doesn’t mean they made the wrong call. The results might not have been what they wanted, but if they believe that a healthy Cutler gives them a better chance to win than McCown would give them, they have to do it. Cutler not playing well wouldn’t automatically mean that McCown would have been better. But let’s hope we won’t have to worry about that particular narrative twist.

Anyway, it’s a good problem to have, isn’t it? Beats the hell out of debating Grossman vs. Orton.

written by

Brett Taylor is the Lead Cubs Writer at Bleacher Nation, and you can find him on Twitter at @BleacherNation and on LinkedIn here. Brett is also the founder of Bleacher Nation, which opened up shop in 2008 as an independent blog about the Chicago Cubs. Later growing to incorporate coverage of other Chicago sports, Bleacher Nation is now one of the largest regional sports blogs on the web.

more bears news

Dan Campbell is Happy To See Justin Fields Leave the NFC North

There was no escaping the mixed emotions from Chicago Bears fans when the Justin Fields trade went down earlier this month. But there were none of that from Detroit Lions Head Coach Dan Campbell. Heck, I'd argue there was just...

Another Pass Rusher Signs Elsewhere: Jadeveon Clowney Lands With the Panthers

Another day. Another pass rusher signing. And another defensive lineman going to a team that isn't the Chicago Bears. NFL Network insider Ian Rapoport was the first with the news of Jadeveon Clowney signing with the Carolina Panthers. Meanwhile, ESPN's...

Malik Nabers Might’ve Run His Way Out of the Chicago Bears Draft Spot

Today was LSU's Pro Day and Chicago Bears GM Ryan Poles was scheduled to be in attendance, per The Athletic's Adam Jahns. Assuming Poles made his flight, he got a good look at the likes of quarterback Jayden Daniels, as...

Goodell on Bears Stadium Deal, No to Hard Knocks? More Old Friends Go to Steelers, and Other Bears Bullets

I keep forgetting that I have a Starbucks gift card from Christmas that has gone unused to this point. Maybe I'll remember it next time I go out for coffee. A big shout out to PFF Brad, who made the...

Ryan Poles Was Pissed Off By RGIII’s Comments About Caleb Williams Situation

When I saw them pop up, RGIII's comments suggesting Caleb Williams pull an Eli Manning and refuse to play for the Chicago Bears were met with an eye-roll, audible groan, and immediate dismissal before moving onto more pertinent topics. But...

Bears Stadium Update: Kevin Warren Reiterates Plans to Build in Downtown Chicago

It sure is starting to sound like Arlington Heights fumbled the bag when it comes to locking in Arlington Park as the future home of a Chicago Bears stadium. While meeting with the media at the NFL owners meetings in...

The NFL is Playing a Christmas Day Doubleheader in 2024

Christmastime is generally full of cheer. And in 2024, it will be full of cheering. In fact, we're talking double the cheer. ESPN NFL insider Adam Schefter reports the league will play not one, but *TWO* Christmas Day games. It's...

2024 NFL Mock Draft: How High Can J.J. McCarthy’s Stock Rise?

Hard to believe we're less than a month away from the 1st Round of the 2024 NFL Draft! With just weeks remaining until the NFL's most noteworthy offseason event, there are plenty of rumors floating. As we work through all...

NFL Rules Shakeup, Tomlin on Fields, Trubisky’s Linsanity Run, and Other Bears Bullets

The Bachelor's season-finale had a couple of jaw-dropping moments. Once the group chat catches up, it is going to be lit. What better way to start Bears Bullets than with breakfast anecdotes? I think I'm going to enjoy D'Andre Swift's...

Around the NFL: New Kickoff Rules Explained, NFL Trade Deadline Extended, More

NFL owners ratified a sweeping overhaul of the kickoff play on Tuesday, embracing a format inspired by the XFL following three days of deliberations at the league’s yearly gathering. Essentially, the new format relocates the bulk of both kicking and return teams...

Latest News

Orioles vs. Angels Prediction: Expert Picks, Odds, Stats & Best Bets – Thursday, March 28, 2024

The Baltimore Orioles begin their season as -192 favorites in a home tilt against the Los Angeles Angels on Thursday at 3:05 PM ET, live on BSW.Baltimore's Corbin Burnes and Los Angeles' Patrick Sandoval have been named the starters for...

Wyoming vs South Dakota Women’s College Basketball Odds and Prediction – March 29

The South Dakota Coyotes (23-12) will try to break a three-game road losing streak when they square off against the Wyoming Cowgirls (17-14) on March 29, 2024 at Arena-Auditorium.Wyoming vs. South Dakota NCAA Game InfoDate: Friday, March 29, 2024Time: 8:30...

Minnesota vs North Dakota State Women’s College Basketball Odds and Prediction – March 29

The Minnesota Golden Gophers (17-15) face the North Dakota State Bison (22-11) on March 29, 2024.Minnesota vs. North Dakota State NCAA Game InfoDate: Friday, March 29, 2024Time: 8:00 PM ETWhere: Minneapolis, MinnesotaVenue: Williams ArenaLast Game For Each TeamMinnesota enters this...

Rays vs. Blue Jays Probable Starting Pitchers – March 28

The Tampa Bay Rays (0-0) and the Toronto Blue Jays (0-0) face off on Thursday at Tropicana Field for Opening Day. First pitch is at 4:10 PM ET.The Rays will call on Zach Eflin versus the Blue Jays and Jose...

Dodgers vs. Cardinals Probable Starting Pitchers – March 28

The Los Angeles Dodgers (1-1) and the St. Louis Cardinals (0-0) face off at Dodger Stadium on Thursday at 4:10 PM ET, starting a four-game series.The Dodgers will look to Tyler Glasnow versus the Cardinals and Miles Mikolas.Dodgers vs. Cardinals...

Padres vs. Giants Probable Starting Pitchers – March 28

The San Francisco Giants (0-0) begin a four-game series against the San Diego Padres (1-1) on Thursday at PETCO Park. First pitch is at 4:10 PM.The Padres will look to Yu Darvish versus the Giants and Logan Webb.Padres vs. Giants...

Astros vs. Yankees Probable Starting Pitchers – March 28

The Houston Astros (0-0) and the New York Yankees (0-0) face off on Thursday at Minute Maid Park for Opening Day. First pitch is at 4:10 PM ET.The Astros will call on Framber Valdez against the Yankees and Nestor Cortes...

Reds vs. Nationals Probable Starting Pitchers – March 28

The Cincinnati Reds (0-0) and the Washington Nationals (0-0) face off on Thursday at Great American Ball Park for Opening Day. First pitch is at 4:10 PM ET.The Reds will call on Frankie Montas against the Nationals and Josiah Gray.Reds...

Marlins vs. Pirates Probable Starting Pitchers – March 28

The Miami Marlins (0-0) and the Pittsburgh Pirates (0-0) match up on Thursday at LoanDepot park for Opening Day. First pitch is at 4:10 PM ET.The Marlins will look to Jesus Luzardo versus the Pirates and Mitch Keller.Marlins vs. Pirates...

Twins vs. Royals Probable Starting Pitchers – March 28

The Kansas City Royals (0-0) host the Minnesota Twins (0-0) in an Opening Day contest at 4:10 PM ET on Thursday.The Twins will call on Pablo Lopez versus the Royals and Cole Ragans.Twins vs. Royals Pitcher Matchup and Game InfoWhen:...

more bears news