A Progressive Tournament Design

[Spit-wad]

Article Link - A Progressive Tournament Design

Single-elimination, double-elimination, group stages (round-robin), best of 3.

Tournaments can have many formats. Multiple formats are often combined together. The goal of tournament design is to ultimately produce the "best winner". Any tournament that has prizes for placing something other than first also needs to yield proper final rankings. Additionally, tournament design needs to provide an exciting competition for spectators.

Unfortunately, eSports has lost sight of these goals, and has adopted a few bad habits.

This article is not meant to target any one specific tournament. The professional DotA scene has been collectively plagued by poor tournament design, and most tournament organizers seem afraid or unwilling to fix the issue at hand. After all, change requires effort (and no, this is not a political appeal). The current standard design often results in tiebreakers that just don't make sense.
<h2>Ties and Tiebreakers</h2>
Tiebreakers in all*forms of competition should be avoided. Tiebreakers are a last resort for a reason: they are not fair. Ties are the result of two or more teams being of a similar caliber, but the competition does not allow for enough detail to differentiate between the teams.

Some of you might remember that I like to compare eSports to professional athletic sports, such as the NFL. American Football is designed in a way that allows ties. At the end of regulation time, the score can be even. After this, there is an overtime session to attempt to create an uneven score. If the score is still even after overtime, the match is simply marked as a tie. Later, for seeding into the playoffs, there is a whole chain of tiebreakers. Head-to-head record, division record, common games record, conference record, strength of victory, strength of schedule, etc. Total points scored throughout the season also makes an appearance on the tie-breaking procedures.

Brace yourselves: I'm about to criticize the NFL (but I won't offer a solution to their problems because this article focuses on DotA). The NFL's*tie-breaking*system isn't terrible, but it*does have its flaws. This is the inherent nature of tie-breaking. The game allows for a tie, so the league*is forced to break it. In DotA, we fortunately do not have this problem. Every game will have a winner and a loser, which means that the issue lies with the way our competitions are designed, not the game itself.

In the NFL, tiebreakers are an unfortunate necessity. At the end of the season, teams can share equal records, and suddenly it becomes important which*games you won, or how you won them. You'll find that the NFL favors the former, and for good reason. How*you win a game should not matter. Total points? The goal of competition is simply to win, not to win with style. Total points is used as a tiebreaker, yet it is considered poor sportsmanship to "drive up the score". After it is obvious who will win the game, the winning team is expected to stop trying as hard.

In DotA, winning the game at 90 minutes with a maxed Spectre should be worth the same as winning the game in 25 minutes with a Pugna push lineup, yet the amount of time taken to win a game is the only tiebreaker our tournaments ever use. I'll say this again: tiebreakers are inherently flawed.

Lucky for us, we can do away with such futile rules.
<h2>Group Stages</h2>
We've all seen tournaments with group stages. Let's take a very common example: 16 teams, 4 groups.
<table>
<tbody>
<tr>
<th>Group 1</th>
<th>Group 2</th>
<th>Group 3</th>
<th>Group 4</th>
</tr>
<tr>
<td>Team A</td>
<td>Team E</td>
<td>Team I</td>
<td>Team M</td>
</tr>
<tr>
<td>Team B</td>
<td>Team F</td>
<td>Team J</td>
<td>Team N</td>
</tr>
<tr>
<td>Team C</td>
<td>Team G</td>
<td>Team K</td>
<td>Team O</td>
</tr>
<tr>
<td>Team D</td>
<td>Team H</td>
<td>Team L</td>
<td>Team P</td>
</tr>
</tbody>
</table>
Each team will play the rest of their teams in the group. Team A will play against teams B, C, and D, etc. Each team plays a total of 3 matches in the group stage. After this, the group stage records will be used to seed 4, 8, or 16 teams into a playoff bracket.

The group stage format has two purposes:

1. Give everyone a chance to play a few games. When traveling to a LAN tournament, nobody wants to play one game, lose, and be eliminated right away. The above example gives each team the opportunity play at least three games.

2. Seeds for a playoff bracket. When starting a new tournament, it's unfair to base seeds on previous performances in other leagues, especially when rosters have likely changed.

Group stages are still created with a small bias by "stacking" certain groups full of better teams, but the bias is ultimately minimized, especially if more teams from each group progress to the playoffs. If there are 8 teams who are better than the other 8 teams, then they would ideally be distributed with two of the best teams in each group. With this format, if 8 teams progress to the playoffs (two from each group), then the "correct" teams would advance.
<h2>The Problem</h2>
<table>
<tbody>
<tr>
<th>Group 1</th>
</tr>
<tr>
<td>Team A (2-1)</td>
</tr>
<tr>
<td>Team B (2-1)</td>
</tr>
<tr>
<td>Team C (2-1)</td>
</tr>
<tr>
<td>Team D (0-3)</td>
</tr>
</tbody>
</table>
Ties. These ties do not exist because of the game itself - they exist because of the tournament design. Many tournaments have this exact setup with four groups, and we almost always see this three-way tie in one of them. When this happens, there are two tie-breaking options:

1. Make them all play again (time consuming, and could result in another tie).

2. Look at how*they won. For some reason our common design has determined that it is fairer to measure time instead of kills as an appropriate tiebreaker. I'm not sure why this is the case, but it's not important. We don't need either one.

First of all, I would like to point out that brackets are designed to specifically prevent this problem. If you think that brackets allow too much variance for "flukes", then use a double-elimination bracket. However, I'm not naive enough to think that tournament organizers will simply drop the group stage format. After all, it does serve a purpose. If we can't remove it, then we should at least try to fix it.

Math time. A tie is a false inequality of some sort. False inequalities occur with smaller numbers, smaller samples, etc.

Flip a coin a few times. You could very easily get 3 heads and 0 tails. Statistics dictates that the more you flip it, the closer you will get to the true 1:1 ratio.*Significant digits is another case. 1.4 ? 1, unless the significant digits deemed relevant by some party say that 1.4 does in fact = 1. More significant digits results in more precision. There are thousands more examples of this in nature and science, but I think you get the gist.

Let's compare a group with three teams to a group with four teams. Assume that Team A > Team B > Team C > Team D, and that if given enough matches, they would be properly ranked.
<table>
<tbody>
<tr>
<th>Group 1</th>
</tr>
<tr>
<td>Team A (1-1)</td>
</tr>
<tr>
<td>Team B (1-1)</td>
</tr>
<tr>
<td>Team C (1-1)</td>
</tr>
</tbody>
</table>
Team A should theoretically go 2-0, but they only have to have one "fluke" game to end up 1-1. For the sake of ties, let's say that Team C also beat Team B in a fluke game, resulting in every team being 1-1.
<table>
<tbody>
<tr>
<th>Group 1</th>
</tr>
<tr>
<td>Team A (2-1)</td>
</tr>
<tr>
<td>Team B (2-1)</td>
</tr>
<tr>
<td>Team C (2-1)</td>
</tr>
<tr>
<td>Team D (0-3)</td>
</tr>
</tbody>
</table>
Team A should theoretically go 3-0. One fluke game still gives them a record of 2-1, which is better than their record of 1-1 in the previous example. If Team C gets their expected win against Team D and also gets a fluke win against teams A or B, then a three-way tie occurs.

One could argue that in this example, these are not "fluke" wins, but rather evidence of counter-play. Team A is not necessarily better than teams B or C. Their play style is effective against team B, but ineffective against team C.

A > B *// *B > C *// *C > A

Even if this is the case, we can fix it! (but for the rest of this article, let's assume that a fluke win occurs when the worse team wins, where the worse team would build a larger losing record as more games are played)
<h2>The Solution</h2>
Bigger groups.

I will use The International as an example, because even Valve makes mistakes (gasp).
<p style="text-align: center;"></p>
The International had three groups with expected outcomes: Groups A, B, and D. The tournament design put the top two teams from each group into a winner's bracket and the bottom two into a loser's bracket. This was a double-elimination bracket, so starting in the loser's bracket equated to being given an extra loss. For the record, m5 went on to win $10,000 more than MiTH-Trust, who was placed into the loser's bracket with the same group stage record, and MYM won even more. Was this a result of an accurate tiebreaker, or the disadvantage of being placed into the loser's bracket? Nobody can say who was truly the "better" team of the three, but MYM's success in the tournament could suggest that their only loss against m5 was perhaps a "fluke", which would have resulted in m5 being 2-1 and starting in the loser's bracket instead of MiTH-Trust. If any MiTH-Trust fans read this: try not to dwell too much on that extra $10,000 they could have won in m5's place. It's a strange world out there, and theory is not always reality.

Let's start with the basics. 16 teams. Why 16? The tournament organizers wanted all teams to enter the bracket stage, and brackets function best with binary numbers (4, 8, 16, 32, etc). For the sake of competition, 16 teams turned out to be a pretty good number. Almost every game was of similar caliber, and was exciting to watch. However, if they weren't restricted by the number of teams being sent to the brackets, the tournament probably would have been just as successful with anywhere from 12-20 teams. If not every team goes to the bracket stage, there is no reason for the total group stage participants to be binary - only the output to the brackets.

For better tournament design, as already stated, we need bigger numbers. Increase the teams in each group, and decrease the winner output to compensate. Please note that this could be accomplished in some form with any number of teams - it just requires a little manipulation.

20 teams:
<table>
<tbody>
<tr>
<th>Group 1</th>
<th>Group 2</th>
<th>Group 3</th>
<th>Group 4</th>
</tr>
<tr>
<td>Team A</td>
<td>Team F</td>
<td>Team K</td>
<td>Team P</td>
</tr>
<tr>
<td>Team B</td>
<td>Team G</td>
<td>Team L</td>
<td>Team Q</td>
</tr>
<tr>
<td>Team C</td>
<td>Team H</td>
<td>Team M</td>
<td>Team R</td>
</tr>
<tr>
<td>Team D</td>
<td>Team I</td>
<td>Team N</td>
<td>Team S</td>
</tr>
<tr>
<td>Team E</td>
<td>Team J</td>
<td>Team O</td>
<td>Team T</td>
</tr>
</tbody>
</table>
There are still four groups, but each one has five teams in it. More teams will reduce the probability of ties from flukes, because there are more matches. Instead of sending all of the teams to the bracket stage, only the top two in each group will advance, and the bracket will only include eight teams. This will still result in roughly the same number of games being played.

12 teams:
<table>
<tbody>
<tr>
<th>Group 1</th>
<th>Group 2</th>
</tr>
<tr>
<td>Team A</td>
<td>Team G</td>
</tr>
<tr>
<td>Team B</td>
<td>Team H</td>
</tr>
<tr>
<td>Team C</td>
<td>Team I</td>
</tr>
<tr>
<td>Team D</td>
<td>Team J</td>
</tr>
<tr>
<td>Team E</td>
<td>Team K</td>
</tr>
<tr>
<td>Team F</td>
<td>Team L</td>
</tr>
</tbody>
</table>
Two groups, six teams each. Seed the top four from each group into an eight-team bracket. If you want fewer matches, then just seed the top two teams from each group into a four-team bracket. This significantly lowers the chances of ties between teams who are close to making it to the following bracket round. Even if there are ties, there's a better chance that it will just be between two teams. If two teams have the same record, their head-to-head match result can be used as a tiebreaker, as this is widely considered the fairest tiebreaker among sports worldwide.

Group stages are the flaw in tournaments. If you need*to have them, reduce that flaw by having more*group stage matches. If you need to compensate for the extra matches, then reduce the amount of matches in other rounds (by reducing the number of teams who make the playoffs, for example). 16 teams is a bad number for a tournament if there are group stages, but it's perfect*for a bracket. There was no reason that 16 teams had to be invited to The International, especially when two had issues and were replaced anyways. It could have been a very smooth-running 12-14 team tournament.

But sometimes you just have to make do with whatever is thrown at you. How do you handle a 13-team tournament? Take the essence of the group stage, but make it just one group. Multiple groups is just the product of an arbitrary or biased seeding to begin with. Instead of using seeds to create groups, you can use real results. Don't use pre-determined matches. Pair the best teams against each other and the worst teams against each other, with the rankings refreshing every round. The first round just uses random pairings, and no pairing is ever repeated. Since we have an odd number of teams, one team just sits out every round. If you haven't seen this in practice, here's an example (this assumes the better team always wins, with the higher letter team being the better team):
<div style="font-size: 10px;">
<table>
<tbody>
<tr>
<th>Round 1</th>
<th>Results</th>
</tr>
<tr>
<td>AvG, BvD, CvI, EvM, FvL, HvJ,*K</td>
<td><span style="color: #888888;">A1-0* B1-0 *C1-0 *E1-0 *F1-0 *H1-0 *K0-0 *G0-1 *I0-1 *J0-1 *L0-1 *M0-1**D0-1</span></td>
</tr>
<tr>
<th>Round 2</th>
<th>Results</th>
</tr>
<tr>
<td>AvC, BvF, EvH, GvK, IvJ, LvM, D</td>
<td><span style="color: #888888;">A2-0**B2-0 *E2-0 *C1-1 *F1-1 *G1-1 *H1-1 *I1-1 *L1-1 *D0-1 *K0-1 *J0-2 *M0-2</span>
</td>
</tr>
<tr>
<th>Round 3</th>
<th>Results</th>
</tr>
<tr>
<td>CvF, GvH, IvL, DvK, AvB, EvJ, M</td>
<td><span style="color: #888888;">A3-0 *E3-0 *B2-1 *C2-1 *I2-1 *G2-1 *D1-1 *F1-2 *H1-2 *L1-2 *K0-2 *M0-2 *J0-3</span>
</td>
</tr>
<tr>
<th>Round 4</th>
<th>Results</th>
</tr>
<tr>
<td>IvG, BvC, AvE, DvF, HvL, KvM, J</td>
<td><span style="color: #008000;">A4-0</span>**<span style="color: #888888;">E3-1 *B3-1 *G3-1 *D2-1 *C2-2 *I2-2 *H2-2 *K1-2 *F1-3 *L1-3 *M0-3 *J0-3</span>
</td>
</tr>
<tr>
<th>Round 5</th>
<th>Results</th>
</tr>
<tr>
<td>EvB, GvD, CvK, IvH, FvL, MvJ, A</td>
<td><span style="color: #008000;">A4-0 *B4-1*</span>*<span style="color: #888888;">D3-1 *E3-2 *C3-2 *G3-2 *H3-2 *F2-3 *I2-3 *K1-3 *J1-3 *L1-4**<span style="color: #008000;">M0-4</span></span>
</td>
</tr>
<tr>
<th>Round 6</th>
<th>Results</th>
</tr>
<tr>
<td>AvM, DvE, BvG, CvH, FvI, JvK, L</td>
<td><span style="color: #008000;">A5-0 *B5-1</span>**D4-1 *C4-2 *<span style="color: #008000;">E3-3</span>**<span style="color: #008000;">G3-3 *H3-3</span>**F3-3 *<span style="color: #008000;">I3-3 *J2-3</span>**K1-4 *L1-4 *<span style="color: #008000;">M0-5</span>
</td>
</tr>
</tbody>
</table>
</div>
This example included just about the worst possible luck with random pairings. As you can see, it only took five rounds to acquire an accurate top eight teams, and after 6 rounds the seeds were almost completely accurate after including head-to-head tiebreakers. With good to average luck, it would be completely accurate after 6 rounds. Regardless, the correct top eight teams were determined for progressing to the playoff bracket. The beauty if this design is that it can be used for any number of teams. (For the record, I'm pretty sure that this is some form of a sorting algorithm, for you Computer Science majors out there.)

The point is that there are*other tournament format options, each of which is suitable for a different number of teams and time constraints (total matches played). These ridiculous tiebreakers to proceed to the next round are the result of poor planning, and they are avoidable.
<h2>Your Fatuous Counter-argument</h2>
"Omg, mTw vs. The Retry in the BenQ DotA2 Clash #2 was so epic! Time limits are awesome, and this wouldn't have happened without tiebreakers!"

Please. If you want to see the game played under a time restriction, then change the game. Leagues use captains mode, which is universally accepted as the fairest form of competition. Years and years of balance have specifically tailored the game for the win condition: win. The game is not balanced if it favors winning faster.

I have always firmly stood by the often misunderstood quote, "Hate the game, not the player". The game is what it is. It is the player's (or team's) job to execute their skills in the game in its current form. Winning with a "cheap" tactic is still winning. If there is an overwhelming opposition to some unfavorable strategies, then the game should be adjusted to compensate for that. If you don't like to see Spectre farming for 90 minutes, then nerf Spectre. Unless the competitive community agrees that winning a game faster should be worth more, then it is not a fair tiebreaker.

Oh, you want me to take this one step further? Alright. If you actually want to see teams playing under time restrictions, then competitive play could have a real final score other than 1-0. Every game is worth 120 points. If a team wins in 0 minutes, they win 120-0. If a team wins in 40 minutes, they win 80-40. There are unlimited variations to a scoring system like this, but they're all equally as unfit for competitive play.

If the tiebreaker is not fit for fair play, then remove the tiebreaker. Group Stages are the source of these ties, but if tournament organizers insist on using them, then at least reduce the chance of ties by using larger groups.

Comment on this article

[ward3n]

There's no doubt that a time-based tiebreaker is pretty much the dumbest thing ever. I wonder how the efficiency of your random-pairing would be affected by increased chance of fluke wins or by counter play. With random pairings, there's variance from pairings and variance from flukes, and it might all add up to result in it being less efficient than some other method in a real-world scenario. I think there's a really high occurrence of "flukes" in dota, mostly because so much depends on picks. It would be interesting to have a tool that could simulate outcomes of tournaments if you gave it data about every team's head-to-head chances versus each other, because it might some sort of emergent behaviors. That said, your format seems pretty good, definitely an improvement.

[Spit-wad]

There's no doubt that a time-based tiebreaker is pretty much the dumbest thing ever. I wonder how the efficiency of your random-pairing would be affected by increased chance of fluke wins or by counter play. With random pairings, there's variance from pairings and variance from flukes, and it might all add up to result in it being less efficient than some other method in a real-world scenario. I think there's a really high occurrence of "flukes" in dota, mostly because so much depends on picks. It would be interesting to have a tool that could simulate outcomes of tournaments if you gave it data about every team's head-to-head chances versus each other, because it might some sort of emergent behaviors. That said, your format seems pretty good, definitely an improvement.

I agree, and that will always be an issue. We can't stop fluke wins from occurring - we can only choose their impact. Every time a tournament uses a group stage, they are using it to seed into a playoff bracket. If there are more group stage matches, it minimizes the impact of fluke wins, which will result in a more accurate seeding for the brackets.

Still, this is exactly why league formats work better. If you have the time to let every team play every other team, then your seeds are theoretically perfect, and any fluke wins will be adjusted in the second round of competition in the bracket stage. In a tournament format (especially LAN tournaments), there simply isn't time to let every team play each other once before the brackets. Pairing up matches by similar rankings with a reset between each round is the closest solution I've found. As previously stated, there are other designs that could work, too... we just need to stop using this groups of 4 setup.

[dasaniwater]

no one is going to read all of that..

[brbclicking]

Posted this on Reddit:

I've been involved in live tournament organization (host for various national events, with 200+ participants) and have been running numbers on these kind of things. I used to love the idea of 5-team groups as well, but trying it in practice and feeling its conquenses, its something that I would do again (at least not in a game like Dota where you have the risk of long-ass games).

The entire article seems to neglect the difference in # of games played in 5-man groups vs 4-man groups, which is the main reason that it isn't used. The 5-man groups is something that every tournament admin have thought of, but the majority are opting out of this.

Its 4 more games pr group (4 man groups = 6 games, 5 man groups = 10 games pr group). A HUGE increase in a game like Dota, where you can get "unlucky" with 90 min games. Schedules are already killing LAN events and everyone from competitors to organizers to stream-viewers hates inaccurate schedules.

As the article points out, group stages are necessary in a tournament. From an admins POV its a bitch: You want for everyone to have at least a couple of games to eliminate as much randomness as possible, but you hate to see your precious time being wasted on teams that shouldn't have been there in the first place. Every tournament has these teams and with 5-man groups you will be wasting an additional hour of a full station (10 PCs) in setup time + game + buffer. The number of stations is your bottleneck in the groupstages (and the early knockout phase), so you want to optimize this as much as possible versus ensuring that everyone "gets a fair chance".

The 6-team groups would equal 15 (!) games pr group, ending up in a total of 30 games for the two groups. That is more games than the 16 team tournament with 4-team groups (a total of 24 games). Thats 6+ hours "saved" on inviting more teams (more exposure), but at the cost of a more "unfair" group-stage. This is a tradeoff almost any admin would take.

Time is the biggest factor at a LAN tournament and you would much rather invest more games into the later brackets (BO3 vs BO5 etc), than you would have more games in the group-stages. The value is simply far, far greater!

From an organizers POV the time-tiebreaker is awesome! Both because it can create some intense matches, but even more-so because it forces the teams to finish early and no dick around. This is the wet dream of any tournament host, as scheduling becomes much easier/safer. Balance and everything aside, you want to avoid 90 min games in the group-stages.

tl;dr: 5-team groups aren't used because its too time-consuming and doesn't add enough value to the LAN tournament.

[Legato]

so based on my skimming what you're suggesting spit is that tournaments go to swiss play instead of a group stage

which I totally and completely agree with

you have swiss play for the first x rounds (whatever number is necessary to definitively determine a top 8/16/32 [however many you want in your playoff bracket]) and then cut to that top number

[Spit-wad]

I'm not suggesting that all tournaments use a Swiss format, I'm just showing that there are better options.

[Legato]

group stages are for wusses/soccer

[HmmMilkshake]

Read most of this. Well thought out and I couldn't agree more. GJ Spitwad. Now hopefully people will listen..?

[Legato]

Read most of this. Well thought out and I couldn't agree more. GJ Spitwad. Now hopefully people will listen..?

I'm Ron Burgundy?

[Scary.]

Oh, you want me to take this one step further? Alright. If you actually want to see teams playing under time restrictions, then competitive play could have a real final score other than 1-0. Every game is worth 120 points. If a team wins in 0 minutes, they win 120-0. If a team wins in 40 minutes, they win 80-40. There are unlimited variations to a scoring system like this, but they're all equally as unfit for competitive play.

I would sort of be okay with this (though probably not that exact scoring system, I see no reason to implement a scoring system which has a bound, in this case, 2 hours), even though it would mean a huge shift in the current meta-game. It would greatly encourage teams to pursue risky fast strategies over the "safe" "asian style" ultra-passive ricing. Further, the time limit would add a lot of tension and add to the watch-ability of the sport for spectators who aren't as familiar with the game as we hardened vets are.

I mean, either way, a slow win would still be better than a loss, and isn't being successful, quickly, harder? I mean push strats are obviously already popular, but. . .

[Legato]

Oh, you want me to take this one step further? Alright. If you actually want to see teams playing under time restrictions, then competitive play could have a real final score other than 1-0. Every game is worth 120 points. If a team wins in 0 minutes, they win 120-0. If a team wins in 40 minutes, they win 80-40. There are unlimited variations to a scoring system like this, but they're all equally as unfit for competitive play.

I would sort of be okay with this (though probably not that exact scoring system, I see no reason to implement a scoring system which has a bound, in this case, 2 hours), even though it would mean a huge shift in the current meta-game. It would greatly encourage teams to pursue risky fast strategies over the "safe" "asian style" ultra-passive ricing. Further, the time limit would add a lot of tension and add to the watch-ability of the sport for spectators who aren't as familiar with the game as we hardened vets are.

I mean, either way, a slow win would still be better than a loss, and isn't being successful, quickly, harder? I mean push strats are obviously already popular, but. . .

uhh

have you watched asian dota recently

[Scary.]

Yeah, it is very push oriented and the games are short. I know. Nothing wrong with even more metagame encouragement for that style imho.

[Sentinelstrike]

Formulating a tiebreaker is hard, whatever you choose people will complain lol.
Could just do a 1v1 game that will take 10 minutes as well, but there would have to be some rulings on that made as well....

[Legato]

Formulating a tiebreaker is hard, whatever you choose people will complain lol.
Could just do a 1v1 game that will take 10 minutes as well, but there would have to be some rulings on that made as well....

haha actually

1v1 game sounds like a fun idea

[m0rphl1ng]

I feel like this needs to be bumped.