

If you've ever taken part in the armchair sport of fantasy football and found yourself at the top of your league's standings at the end of the season, a new MIT study suggests your performance—however far removed from any actual playing field—was likely based on skill rather than luck.

Those looking for ways to improve their fantasy will have to look elsewhere: The study doesn't identify any specific qualities that make one fantasy player more skilled over another. Instead, the researchers found, based on the win/loss records of thousands of fantasy players over multiple seasons, that the game of fantasy football is inherently a contest that rewards skill.

"Some [fantasy] players may know more about statistics, rules of the game, which players are injured, effects of weather, and a host of other factors that make them better at picking players—that's the skill in fantasy sports," says Anette "Peko" Hosoi, associate dean of engineering at MIT. "We ask, does that skill have an impact on the outcome of the [fantasy] game? In our analysis, the signal for skill in the data is very clear."


"They probably can't use our study to assemble better sports teams," says Hosoi, who is also the Neil and Jane Pappalardo Professor of Mechanical Engineering. "But they can use it to talk better smack when they're at the top of their standings."

The team's findings appear this week in the Society for Industrial and Applied Mathematics Review. Hosoi's co-authors are first author Daniel Getty, a in MIT's Department of Aeronautics and Astronautics; graduate student Hao Li; former graduate student Charles Gao; and Masayuki Yano of the University of Toronto.

A fantasy gamble



However, the UIGEA was not drafted to alter the legality of internet wagering, which is, for the most part, determined by individual states. As fantasy sports—and in particular—have grown more popular, with prominent ads on commercial and cable television, a handful of states have questioned the legality of fantasy sports and the companies that enable them.


"That is the question that FanDuel wanted us to investigate: Have they designed the contest such that skill is rewarded? If so, then these contests should be classified as games of skill, and are not gambling," Hosoi says. "They gave us all of their data, and asked whether we could determine the relative role of skill and luck in the outcomes."

Tests of skill and chance


They reasoned that if a game were more skill-based, then a player's performance should be persistent. It might be good or bad, but it would remain relatively constant over multiple rounds.

To test this in the context of fantasy sports, Hosoi's team looked at the win/loss record of every fantasy player in FanDuel's dataset, over one season. For each (anonymized) player, the researchers calculated the fraction of wins the player experienced over the first half of the season versus the second half. They then represented each player's performance over an entire season as a single dot on a graph whose vertical and horizontal axes represented the win fraction for the first and second halves of the season, respectively.

If a given fantasy sport were based more on skill, then a individual player's win fraction should be approximately the same—be it 90 percent or 10 percent—for the first and second halves of the season. When every player's performance is plotted on the same graph, it should roughly resemble a line, indicating a prevalence of skill. On the other hand, if the game were one of chance, every player should have around a 50 percent win fraction, which on the graph would look more like a circular cloud.



"Even when you correct for biases, like people who quit after losing a lot of games in a row, you find there's a statistically higher win fraction for people who play a lot versus a little, regardless of the [type of] fantasy sport, which is indicative of skill," Hosoi says.

The last test, again proposed by Levitt, was to see whether a player's actions had any impact on the game's outcome. If the answer is yes, then the game must be one of skill.

"So we looked at how the actual playing population on FanDuel performed, versus a random algorithm," Hosoi says.


"We ran hundreds of thousands of games, and looked at the scores of actual fantasy players, versus scores of computer-generated fantasy players," Hosoi says. "And you see again that the fantasy players beat the computer-generated ones, indicating that there must be some skill involved."

Sports on a spectrum

To put their findings in perspective, the researchers plotted the results of each fantasy sport on a spectrum of luck versus skill. Along this spectrum, they also included each fantasy sport's real counterpart, along with other activities, such as coin flipping, based entirely on chance, and cyclocross racing, which hinges almost entirely on skill.



"That's a lot of games, and there are a lot of scoring opportunities in each game," Hosoi says. "If you get a lucky basket, it doesn't matter too much. Whereas in hockey, there are so few scoring opportunities that if you get a lucky goal it makes a difference, and luck can play a much larger role."

Hosoi says the team's results will ultimately be useful in characterizing sports, both in and out of the legal system.

"This is one piece of evidence [courts] have to weigh," Hosoi says. "What I can give them is a quantitative analysis of where [] sit on the skill/luck spectrum. It's mostly skill, but there's always a little bit of luck."

More information: Daniel Getty et al. Luck and the Law: Quantifying Chance in Fantasy Sports and Other Contests, SIAM Review (2018). DOI: 10.1137/16M1102094