Fashion and analysis on Project Runway

[#1]

As a Project Runway fan, I wondered: Is it better to win early or later? How does an early win relate to a contestant’s ultimate success? How often do winners NOT win their first round until later in the season? (Photo courtesy of Marsha Alford)A few years ago, my wife started watching Project Runway and quickly became hooked. She has since watched every single episode of the TV series, including Project Runway Junior and Project Runway All-Stars. This year for her birthday, we even went to New York Fashion Week to see some actual shows.

Therefore, I have seen more than a handful of episodes.

On a typical season, 16 designers are given a challenge, a set budget for fabric (“Thank you, Mood!”), and one or two days to create a garment. Afterward, the completed creations are presented to the judges in a runway show. The designer with the highest score wins the round. The designer with the lowest score is eliminated. For the final round, three or four designers are tasked with creating a collection, which are then presented in the final runway show, usually at New York Fashion Week. The ultimate winner of 14 rounds receives a huge prize package and, hopefully, a significant career boost.

Because my wife has watched many of the seasons in reverse order, we already knew who a lot of the season winners were before she got to the older episodes. I noticed that some these season winners didn’t get their first win until late in the competition. For example, Season 6 winner Irina Shabayeva didn’t have a challenge win until episode 5. And Dmitry Sholokhov, who won both Season 10 and All-Stars Season 4, didn’t have his first challenge win until episode 9.

This got me to thinking:

• Is it better to win early or later?
• How does an early win relate to a contestant’s ultimate success on the show?
• How often do winners NOT win their first round until later in the season?

I thought it would be fun to do some analysis in JMP to find some answers. So I teamed up with resident statistician @clay_barker. He created a data table that tracked each designer who won the first episode, that designer’s final appearance, total number of episodes, rank, and whether or not the designer made it to the finale.

Our source data came from Wikipedia, which charts the wins, losses, and eliminations for each season. Because Season 17 was currently running at the time of our analysis (it concluded on June 13, 2019), we only used data from the first 16 seasons of the show.

Clay offered that because it’s a competition where one person’s outcome impacts every other designer’s outcome, it was tricky to do more than some basic tests. The same was true for there being a different number of contestants in some seasons. But this still led to some interesting results.

Is it actually better to lose the first round?

There are two ways to look at the first question. On the surface, it appears that not winning the first challenge is better. Out of 16 seasons, only three designers have won both the first challenge and the whole season, which comes to a success rate of less than 20% (this held true for Season 17).

However, if a designer does win the first competition, all is not lost.

As Clay discovered, designers who win the first challenge are more likely to win the season than if everyone had equal probability. Most seasons have 16 contestants, so if all of the contestants were on equal footing after the first episode, then the remaining contestants would each have a 1/15 (1/15=.06667) chance of winning the whole season. Clay tested the hypothesis that the probability is equal to 1/15. And the p-value was small enough (.0865) to provide pretty strong evidence that people who win the first episode have a probability of winning the season that is greater than 1/15. So they appear to have an advantage compared to if everyone had the same chances.

Another way to think of it is that if everyone were equal and had a 1/15 chance of winning, we’d expect the contestant who won the first challenge to win the whole season 16*(1/15) = 16/15 times because there have been 16 seasons. We’d expect that to have happened once, but it’s actually happened three times.

One expected result from Clay’s analysis was that, of the people who won the first episode, half of them made it to the finale. If winning the first challenge didn’t have impact on a designer’s chances of making it to the finale, we’d expect that the chances would be 3/15=.2. Since that p-value is small, it suggests that designers who win the first challenge have a better shot at making the finale.

This is perfectly illustrated by Clay’s distribution analysis, which shows how long each designer “survived” in the competition. The Event Plot at the top shows when the designer was eliminated. The probability plot below it shows the probability of surviving to a certain point. Clay drew the red reference line at .5 so that we can see that the median survival time is about 13 – in other words, half of the people who win episode 1 go on to survive until episode 13.

I had thought that episode 1 winners tended to go out early, but that is clearly not the case – just the sort of unexpected result I was hoping we got!

This result also held true in Season 17. As I had expected, Tessa Clark, who won the first challenge, did not win the final competition. However, she did last until episode 12, just before the finale. Which perfectly matched Clay’s analysis.

Is it better to win late than to win early?

To explore this question, I followed Clay’s lead and created a new data table. For this one, I recorded each winner, episode of their first win, last win, and total number of wins. Performing a few more simple analyses led to some additional interesting results.

I began with a distribution of first wins by episode. As I suspected, the majority of them were in the second half of the season. The probability of winning the season seemed to increase with later wins, which makes sense considering that the judges would have a more recent view of success and would have seen a trajectory of improvement.

Looking at the data, however, led me to another observation. I noticed that the majority of season winners had more than one episode win under their belt. So, I did another distribution and found that, with only two exceptions (Jay Carroll, 0; Kentaro Kameyama, 1), all of the season winners had won at least two episodes. Just over half (9/16) had won exactly two. Another unexpected result!

How often do winners NOT win their first round until later in the season?

Now to one final analysis: the last win for each season winner. For this I did a scatterplot, where you can see that most season winners had their final episode win later in the season.

Based on these analyses, I came to the conclusion that my original hypothesis was largely correct: In order to win the show, it is better to not win the first episode. And that many season winners do not have their first win until later in the competition.

However, I also learned that most episode 1 winners tend to last through the majority of the season. And that it is still quite possible to both win the first episode and the whole thing.

You can explore the interactive results of this analysis at JMP Public.

Update 2/28/2020: As our analysis predicted, the latest Episode 1 winner, Brittany Allen, did not win Season 18. And even considering her save from elimination in Episode 9, she also made it very far into the competition. Season winner Geoffrey Mac had his first challenge win in Episode 5, with two wins total prior to the finale.

Update 2/5/2022: The same held true for Season 19. Bones Jones won the first challenge, but did not win the season. He did, however, make it to the final elimination (Episode 12), just before the finale (Episode 14). Season winner Shantall Lacayo had four challenge wins; the first in Episode 4 and the remaining three in Episodes 9-12.